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EXECUTIVE SUMMARY 



POWERSOURCE® Background and Rationale 

The POWERS OURCE intervention is intended as a generalizable and powerful 
formative assessment-based strategy that can be integrated with any ongoing mathematics 
curriculum to improve teachers’ knowledge and practice and, in turn, student learning. 
Combining theory and research in cognition, assessment, and learning with design elements 

(E) 

to support the transformation of practice within existing constraints, POWERSOURCE 
includes both a system of learning-based assessments and an infrastructure to support 
teachers’ use of those assessments to improve student learning. The current study focuses on 
middle school mathematics, starting in 6 th grade, and on helping to assure that students 
possess key understandings they need for success in Algebra I. Our primary research 
objectives are based on our hypotheses that as a result of POWERSOURCE , teachers will 
become more proficient in their subject matter knowledge, more skilled in their formative use 
of assessment, and better focus their instruction on key ideas; in turn, teachers will be more 
effective in helping students to improve their understanding — as shown by measures of 
student learning. 

(El 

A striking innovation in POWERSOURCE is its targeting of the big ideas — 
fundamental concepts and principles — and their interrelationships that underlie and define a 
field of knowledge (rather treating specific concepts and topics in isolation, as do 
traditionally developed tests). The POWERSOURCE intervention targets big ideas and 
related skills in four domains underlying success in Algebra I: a) rational number 
equivalence (RNE), b) properties of arithmetic (PA), c) principles for solving linear 
equations (SE) and d) application of core principles in these domains to other critical areas of 
mathematics, such as geometry and probability (RA). These domains were chosen because of 
their importance to later mastery of algebra and their significant place in mathematics 
standards across 6 th through the 8 th grade. 

In each domain we designed a series of short POWERSOURCE® assessments 
comprised of multiple item types, which are called Checks for Understanding , to help 
teachers assess their students’ understanding of basic mathematical principles as well as to 
connect their instruction and provide feedback to support deeper understanding. A set of 
instructional resources and targeted professional development activities were also developed 
for each of these domains. POWERSOURCE materials are designed to complement 
existing curricula. 
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Updated Results from 2006-07 POWERSOURCE® Field Test 

During the 2006-07 school year we conducted two types of inter-related studies 
building on the prior project years’ work: a) continuing investigation of item quality of the 
Checks for Understanding , supplementing data we obtained on items in the previous year; 
and b) experimental tests (using random assignment) of the 6 th grade POWERSOURCE® 
materials in four districts in two states. Analyses of these two types of data were conducted 
during the 2007-08 project year. 

Item Quality Analyses of 2006-07 Field Test Data 

Several kinds of item-level analyses were carried out: confirmatory factor analyses, 
reliability analyses, and Item Response Theory (IRT) analyses. Our typical scheme for 
analyzing each set of Checks for Understanding was to first calculate reliability coefficients 
(Cronbach’s alpha) for the items comprising the set. Second, as another check of item 
quality, we conducted a principal component analysis and a confirmatory factor analysis for 
each test form to check whether the items exhibited the factor structure we expected (e.g., 
whether the computation items loaded on the same factor). Third, IRT analyses based on 
Rasch models, were conducted in order to obtain item parameters (difficulties) and item 
characteristic and information curves so that we could use them to select items for future 
testing. 

Results were generally supportive of the statistical quality of the measures. Although a 
few items appeared not to fit well by Rasch models, this finding does not invalidate the 
measure. It simply indicates that beyond the strong overall achievement measured by each 
domain’s test forms, there are also some minor dimensions of achievement that impact the 
individual item scores of individual students. That the overall dimensions (or principal 
components) measured by each subject assessment are very strong is demonstrated by both 
1) strong Cronbach’s alpha internal consistency reliabilities (a measure of measurement 
precision of the overall dimension derived outside the IRT model) and 2) the positive results 
from the confirmatory factor analysis and principal component analysis. 

POWERSOURCE® Field Test 2006-07: Experimental Comparison Findings 

As noted in the previous annual report, sixty-six 6 th grade teachers were recruited from 
middle- schools in Arizona and California to participate in the 2006-07 field test. Within each 
district, teachers were randomly assigned to experimental (POWERSOURCE ) and 
comparison groups. Experimental group teachers in all cases participated in initial summer 
professional development and after school follow-up sessions; they also used project 
materials, such as the Checks for Understanding as well as instructional supports (including 
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teacher instructional handbooks). Comparison group experiences varied slightly depending 
on district need and configuration, with all comparison group teachers having no access to 

the POWERSOURCE instructional supports. All teachers gave eight Checks for 

( 2 ) 

Understandings throughout the school year — two for each of the four POWERSOURCE 
modules. 

( 2 ) 

Significant differences were found between POWERSOURCE and control students’ 
performance on project-developed measures for all districts and domains. The effect sizes 
ranged from .45 to 3.30 across the four districts and the five content areas, with a median 
effect size of 0.84 (mean effect size is 1.05). The median effect sizes by content area are 
1.12, 0.80, 0.81, and 2.04, respectively for each of the units. The median effect sizes by 
district ranged from 0.54 to 1.01. These findings pertain to POWERSOURCE 0 tasks, which 
are broadly representative of fundamental pre-algebra content. 

In terms of next steps, we are in the process of collecting and analyzing both prior and 
concurrent state standards test data in mathematics for both the POWERSOURCE and 
control groups. The prior year’s data would serve to document the initial comparability of the 
two groups and could be used as a covariate for analysis; the concurrent year data would 
serve as a partial transfer measure. 

As part of the 2006-07 field test, based on an activity that asked teachers to rank a 
series of student assessment responses, we also examined changes in teacher knowledge 
across professional development participation. This activity was integrated into the 
POWERSOURCE professional development meetings at both the beginning and end of the 
2006-07 school year. We analyzed results from this measure to provide some basic statistical 
quality information about the measures and also to assess pre- and post- professional 
development differences in teacher knowledge. 

In summary, we found the teacher knowledge ranking task to be a short, relatively non- 
invasive tool for assessing teacher knowledge that produces relatively reliable results as well 
as a scale that allows for cumulative interpretations. Among 2006-07 teachers who took both 
the pre- and the post- ranking assessment, composite scores demonstrate statistically 
significant and substantively important gains. 

POWERSOURCE® Implementation Study 2007-08 

The core undertaking of our work during the 2007-08 school year was to conduct an 
extended, random assignment implementation study of our 6 th grade POWERSOURCE® 
program. As with the 2006-07 field test, teachers were randomly assigned to either 
POWERSOURCE or control conditions with the ultimate goal of determining program 
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impact on both students and teacher learning outcomes. The 2007-08 study, however, 
differed from the previous year’s work in a number of important ways. Specifically: 

• The experimental design incorporated both within- and between-school random 
assignment models. That is, for some of the districts the random assignment 
accomplished within each school (i.e., a given school had both POWERSOURCE 
and control teachers), and for some the random assignment was between school 
(i.e., all teachers at a given school were POWERSOURCE or control). Ultimately, 
a total of 112 6 th grade teachers across seven school district participated in the 
study. 

(S) 

• Although the content focus of the four POWERSOURCE modules remained the 
same (RNE, PA, SE, and RA), based on teacher feedback and our implementation 
experiences in 2006-07, the structure of each unit somewhat changed. In 2007-08, 
POWERSOURCE teachers were provided with three Checks for Understanding 
for each unit — one prior to the first day’s set of instructional materials, another in 
between the first and second day of instruction, and one after the second day of 
instruction. Thus, the students completed 12 Checks for Understanding (three for 
each of the four units) during the school year. The Checks for Understanding and 
instructional materials were also revised/ refined based on analysis of 2006-07 item 
level data. 

• Unlike 2006-07, the control students did not complete any of the Checks for 
Understanding (i.e., the short formative assessments). Thus, the 2007-08 control 
students and teachers had no exposure to any of the POWERSOURCE materials 
or concepts during the school year. 

• All students (POWERSOURCE® and control) completed a test of prerequisite 
knowledge at the beginning of the school year and a transfer measures of math 
knowledge at the end of the school year. The test of prerequisite knowledge will 
serve as a baseline measure for later analyses; whereas, the transfer measure will 
serve as an independent student outcome measure (in addition to state test data). 

• Based on district response and feedback, districts were offered the option of the 
control teachers receiving an alternative (i.e., non-POWERSOURCE ) professional 
development from the National Center for Research on Evaluation, Standards, and 
Student Testing (CRESST). The majority of participating districts selected this 
option. 

• Additional teacher knowledge measures were included in the study, and a 
qualitative teacher observation/interview pilot study was undertaken. 

Several supplementary strands of work were completed as part of CRESST activities 
during the 2007-08 school year as well. These studies include a validation study of teacher 
math knowledge measures, continued investigation of how computer-based assessment 
strategies can support mathematics learning, investigation of states’ English language learner 
(ELL) assessment and accommodation practices (as part of CRESST supplementary grant), 
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investigation of district contexts for assessment, and international applications of the 
POWERSOURCE® work. 

Plans for 2008-09 

The focus for project implementation during the 2008-09 school year will be continuing 
the experimental (random assignment) study of POWER S O URGE impact begun in 2007- 
OS. Specifically, in addition to continuing the study at the 6 th grade level, we plan to add the 
7 th grade teachers in the participating districts to the study. The study will utilize a similar 
design and instrumentation to what was used in the 2007-08 study — with student and teacher 
outcome instruments adapted to reflect 7 th grade content as applicable. During the 2008-09 
school year we will also develop 8 th grade materials (including Checks for Understanding , 
instructional activities, professional development resources), with the goal of expanding the 
study to 8 th grade in the 2009-10 school year. 

A planned set of supplemental activities for 2008-09 is part of the leadership strand of 
work. Our leadership activities intend to support states and districts in their desire to develop 
coherent instructional programs to engage in standards-based reform. The work focuses in 
two areas. First, it will focus on the collaborative development of methodology and annotated 
examples that practitioners and contractors can use to align instruction and assessment 
developmentally with key priorities for student capability in mathematics as well as with 
standards. Secondly, to support the ongoing quality current assessments, we propose a series 
of working conferences, connected to wider audiences through webinars, to clarify the 
specific criteria that states and local districts and schools should address in selecting and 
refining these assessments and to compare and learn from promising systems for which there 
are positive data. We propose also to make the criteria and examples of their application 
available interactively on the CRESST website. These efforts will concentrate on areas of 
recent CRESST research, formative assessment and English Language Proficiency (ELP) 
assessment, as well as provide a forum for sharing the best of current knowledge from 
CRESST and other leading efforts across the country. 
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THE DEVELOPMENT AND IMPACT OF POWERSOURCE®: YEAR 3 

Julia Phelan, Terry Vendlinski, Kilchan Choi, Joan Herman and Eva L. Baker 
CRESST/University of California, Los Angeles 

Abstract 

The POWERS OURCE© intervention is a generalizable and powerful formative 
assessment-based strategy that can be integrated with any ongoing mathematics 
curriculum to improve teachers’ knowledge and practice and, in turn, student learning. 
Our primary research objectives are based on our hypotheses that as a result of 
POWERSOURCE©, teachers will become more proficient in their subject matter 
knowledge, more skilled in their formative use of assessment, and better focus their 
instruction on key ideas. Ultimately, teachers will be more effective in helping students 
to improve their understanding — as shown by measures of student learning. This report 
provides a detailed overview of our study results from the 2006-07 school year - . We 
found that a knowledge mapping task and a student response analysis task, could be used 
as a set, or the knowledge mapping task could be used separately from the other tasks, in 
order to efficiently investigate both content knowledge of teachers and knowledge of 
teaching algebra. 



Introduction 

The POWERSOURCE intervention is intended as a generalizable and powerful 
formative assessment strategy that can be integrated with any ongoing mathematics 
curriculum to improve teachers’ knowledge and practice and, in turn, student learning. 
Combining theory and research in cognition, assessment and learning (for both adults and 
students) with design elements to support the transformation of practice within existing 
constraints, POWERSOURCE includes both a system of learning-based assessments and an 
infrastructure to support teachers’ use of those assessments to improve student learning. The 
current study focuses on middle school mathematics, starting in 6 th grade, and on helping to 
assure that students possess key understandings they need for success in Algebra I. Such a 
focus is motivated by ample research showing the frequency and price of failure for 
subsequent academic performance, including high school graduation, college entry and 
preparation (e.g., Brown & Niemi, 2007). Our primary research objectives are based on our 
hypotheses that as a result of POWERSOURCE , teachers will become more proficient in 
their subject matter knowledge, more skilled in their formative use of assessment, and better 
focus their instruction on key ideas; as a result, teachers will be more effective in helping 
students to improve their understanding, as shown by measures of student learning. 
Ultimately, we expect the improvements in student understanding to drive better performance 



1 




on No Child Left Behind (NCLB) mandated state tests, transfer measures, and future 
coursework. 

Research on Formative Assessment 

The intervention builds on recent research showing formative assessment as a powerful 
strategy for improving learning. (Black & Wiliam, 1998a, 1998b; Bloom, 1968; Kluger & 
DeNisi, 1996). For example. Black and Wiliam’s (1998a) landmark meta-analysis, based on 
a review of 250 studies, found effect sizes that ranged between .4 and .7, and found 
particularly large effect sizes for low-achieving students — including students with learning 
disabilities (Black and Wiliam, 1998b). This finding makes intuitive sense, as one of the 
major functions of formative assessment is to determine where students are relative to 
learning goals and to use this information to provide feedback and/or make necessary 
instructional adjustments — such as re-teaching, trying alternative instructional approaches, or 
offering more opportunities for practice. If students have already mastered the content, there 
is little need for subsequent adjustment and little room for learning improvement. 

Yet even as research shows the rich potential of formative assessment, so too it 
suggests the limits of current practice. The quality of increasingly popular interim or 
benchmark testing, marketed as formative assessments to districts and schools, is uneven; in 
fact, assessment tends to be an afterthought rather than a core, quality element of current 
curriculum materials (Herman & Baker, 2006; Herman, Osmundson, Ayala, Schneider, & 
Timms, 2006; Wolf, Bixby, Glenn, & Gardner, 1991). Moreover, educators often have 
limited background and capacity to develop or engage in quality assessment practices 
(Heritage & Yeagley, 2005; Herman & Gribbons, 2001; Plake & Impara, 1997; Shepard, 
2001: Stiggins, 2005). For many teachers, their current classroom assessment practices are 
almost exclusively summative, consisting, for example, of end-of-the-week, unit or semester 
tests. 

Students receive grades or scores on these assessments and their teachers — who have 
neither the time nor the curriculum resources to remediate deficiencies — move on, 
disconnecting the assessments from any active function in learning. Yet as Black and Wiliam 
(1998a, 1998b) note, assessments can only become formative when information from them is 
used immediately to inform teaching and for the benefit of student learning. Teacher subject 
matter knowledge offers yet another challenge, as research and our own experiences in 
assessment development with teachers and districts suggest that many teachers do not have 
subject area knowledge sufficiently deep to teach or assess mathematics effectively (Ball & 
Bass, 2001; Ball, Lubienski, & Mewborn, 2001). 
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Learning to use assessment in a more formative way, thus, requires significant changes 
for many districts, teachers, and students. For districts, it will mean ensuring that teachers 
have the time and resources to act on the assessment information they receive. For teachers 
and students, it will involve learning to use assessment information diagnostically to 
determine the course of instruction and learning, and to deal with learning difficulties that are 
revealed by formative assessments. Given the challenges involved in changing assessment 
practices, a substantial part of our research and development, therefore, focuses on exploring 
the types and frequency of assessments and instructional supports that will be feasible to 
implement and most beneficial to teachers and students (e.g., helping teachers to understand 
mathematical concepts more deeply, monitor learning of key ideas and skills, and to figure 
out the best strategies to improve students’ understanding). 

Learning-Based POWERSOURCE® Strategy 

The POWERSOURCE intervention involves not only the development of formative 
assessments, but also the development of professional development and instructional support 
resources to help teachers to understand the mathematical content, interpret assessment 
information, provide feedback to students, and adapt instruction as needed. Moreover, a 
striking innovation in POWERSOURCE is its targeting of the big ideas — fundamental 
concepts and principles — and their interrelationships that underlie and define a field of 
knowledge, rather treating specific concepts and topics in isolation (as do traditionally 
developed tests). This innovation is motivated by ample evidence from a range of cognitive 
psychology perspectives, which suggest that for learning to be acquired efficiently and 
sustained it must enable students to connect to organizing principles what otherwise would 
be disconnected knowledge or procedures and to integrate and demonstrate their knowledge 
and skills in many situations, in near and far transfer, and across time (e.g., Atkinson & 
Shiffrin, 1968; Chi, Feltovich & Glaser, 1981; Ericson, 2003; Ericson & Simon, 1984; 
Hiebert & Carpenter, 1992; Mayer, 2003; Brown, Bransford, & Cocking, 1999; Newell, 
1990, VanLehn, 1996, Catrambone & Holyoak, 1989). 

(G) 

Similarly, the specific item types used in POWERSOURCE were developed based on 
cognitive research demonstrating the value of specific strategies for promoting transfer. 
Research, for example, suggests that learning and problem solving strategies can be 
successfully transferred if students are taught to focus on self-evaluation or metacognition 
(Moreno & Mayer, 2005; Palincsar & Brown, 1984; Pressley & Brainerd, 1985); the 
conditions for applying strategies (Judd, 1908; 1936; Kilpatrick, 1992); the building of 
principled representations of problem situations (Fuchs, Fuchs, Finelli, Courey, & Hamlett, 
2004; Kilpatrick, Swafford, & Findell, 2001); worked-out examples as a way to build 
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problem schemas that generalize across a range of tasks (Chi & Bassok, 1989; Pawley, 
Ayres, Cooper, & Sweller, 2005); and lastly, the explanation and problem solving tasks 
requiring understanding of core concepts and principles that reoccur across arithmetic, pre- 
algebra, and algebra (Carpenter & Franke, 2001; Haverty, 1999; Ready, Edley, & Snow, 
2002; Schmidt, McKnight, & Raizen, 1997). POWER S O U R C E not only uses item types 
that are positioned to uniquely foster learning, but it also purposively employs multiple 
formats to promote transfer, rather than focusing only on those representations adopted by 
test developers designing for accountability purposes (Richardson-Klavehn & Bjork, 2002). 

Targeted Domains Operationalized in Checks for Understanding 

The POWERSOURCE intervention targets big ideas and related skills in four domains 
underlying success in Algebra I: a) rational number equivalence (RNE), b) properties of 
arithmetic (PA; the distributive property), c) principles for solving linear equations (SE); and 
d) application of core principles in these domains to other critical areas of mathematics, such 
as geometry and probability (RA). These domains were chosen because of their importance 
to later mastery of algebra and their significant place in state mathematics standards across 
6 th through 8 th grade. 

In each domain we have designed a series of short POWERSOURCE assessments 
comprised of multiple item types, which are called Checks for Understanding , to help 
teachers assess their students understanding of basic mathematical principles and to connect 
their instruction and provide feedback to support deeper understanding. A set of instructional 
resources and targeted professional development activities were also developed for each of 
these domains. Thus, a POWERSOURCE module around a given domain includes a set of 
Checks for Understanding , targeted instructional resources, and professional development 
opportunities. POWERSOURCE materials are designed to complement existing curricula, 
but time for it must be found within tight district curriculum frameworks and timelines. It is 
thus important for POWERSOURCE to integrate well and easily with existing initiatives, 
not add an unreasonable burden to the heavy testing requirements already imposed on 
teachers (e.g., weeks of state and district testing), and not replace large chunks of extant 
curricula. 

More detailed information about the research foundations, content focus, initial 
development process, and program components of POWERSOURCE can be found in the 
CRESST’s 2006 and 2007 progress reports to the Institute of Education Sciences (Baker, 
2007). The present report focuses on providing an update on project activities undertaken 
since the last progress reporting period (i.e., covering the 2007-08 school year). This update 
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is organized around four general areas. First, we provide updated results from the 2006-07 
experimental (randomized) field test of POWERSOURCE instructional sensitivity — 
including both item quality data for the Checks for Understanding and treatment/control 
differences on student and teacher outcomes. Second, we describe the experimental 
(randomized) study conducted during the 2007-08 school year, for which data collection is in 
its final stages. Third, we describe design and pilot testing of 7 th grade materials conducted 
during the 2007-08 school year. Finally, we provide updates on supplemental/synergistic 
research studies, including technology-based activities and English Language Learner (ELL) 
assessment studies. Dissemination activities are also discussed, as well as an overview of 
planned activities for the 2008-09 school years. 

Updated Results from 2006-07 POWERSOURCE® Field Test 

As described in previous progress reports (Baker, 2007) during the 2006-07 school year 
we conducted two types of inter-related studies building on the prior project years’ work: a) 
continuing investigation of item quality of the Checks for Understanding , supplementing data 
we obtained on items in the previous year; and b) experimental tests (using random 
assignment) of the 6 th grade POWERSOURCE® materials in four districts in two states. 
Detailed description of the methodology used and preliminary results were presented in these 
previous progress reports. Following is an updated summary of 2006-07 school year results 
from these two inter-related strands of work. 

Item Quality Analyses of 2006-07 Field Test Data 

Analyses of item and test data from 2006-07 field test continued and were completed 
over the course of the current year. Several kinds of item-level analyses were carried out: 
confirmatory factor analyses, reliability analyses, and Item Response Theory (IRT) analyses. 
Our typical scheme for analyzing each set of Checks for Understanding was to first calculate 
reliability coefficients (Cronbach’s alpha) for the items comprising the set. Second, as 
another check of item quality, we conducted a principal component analysis and a 
confirmatory factor analysis for each test form to check whether the items exhibited the 
factor structure we expected; for example, whether the computation items loaded on the same 
factor. Third, IRT analyses based on Rasch models were conducted in order to obtain item 
parameters (difficulties) and item characteristic and information curves so that we could use 
them to select items for future testing. The model-data fit was investigated using two model 
fit indices. One is the G~ index which is the Chi-square (y ) statistic and provided in 
PARSCALE phase 2 outputs, and the other is the mean square fit (MNSQ) statistics. 
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As described in our previous progress report (Baker, 2007), we used 2005-06 and 2006- 
07 pilot test information to refine our assessment forms and develop Checks for 
Understanding that then were then administered to students as part of a 2006-07 randomized 
field test. This field test had two major objectives: a) to refine the design of our materials, 
and b) to provide some preliminary information about POWERSOURCE effectiveness. In 
the following section, we provide some highlights of these findings. 

Reliability and factor analysis. Appendix A (Additional Item Analysis Results) 
contains detailed reliability coefficients and factor analysis results for Checks for 
Understanding forms in 2006-07 field testing. Each domain had two test forms, so there were 
total of 10 forms of RNE, PA, SE, RA, and FM. The range of reliability coefficients 
(Cronbach’s alpha) were between 0.554 and 0.863. Based on the inter-correlations between 
the items, factor analysis further determines the theoretical constructs that might be 
represented by the set of items in a form. This analysis allowed us to look at the Checks for 
Understanding forms in each domain and determine whether the items exhibited the factor 
structure we expected. Values less than .05 for the RMSEA indicate a close fit, with values 
as high as .08 representing a reasonable fit (Joreskog & Sorbom, 1993, p. 124). The GFI 
provides a measure of the relative amount of variance and covariance accounted for by the 
model. Values greater than .90 for the GFI measure is required to indicate a good fit (Byrne, 
1994). According to these criteria, the RMSEA and GFI values indicated that the items’ 
variance in each test could be explained well by a single construct. In each form, the main 
component accounted for 27.529% through 50.195% of the total variance, which suggested 
the items measured a unidimensional trait (see Appendix A more detailed results). 

IRT analyses. Appendix A contains item parameters of dichotomous and polytomous 
items estimated using the program PARSCALE as well as the item characteristic curve of 
every item in the five domains. The item parameter calibration for field testing was executed 
for each domain rather than for a form in each domain; whereby, the item parameter 
estimates of a domain could be considered being put onto a common scale. For example, 
item parameter estimates for FM in 06-07 field testing ranged from -2.82 (0.11) to 2.60 
(0.09). 

Using the SE items as an example, (see Figure 1 for item (category) characteristic 
curves and item information curves of items on the 2006-07 SE-11 field test form), SE-11 
has five dichotomous and only one polytomous items. Figures la, lb, and lc respectively 
show item characteristic curves of the dichotomous items (SE-BT-7, SE-BT-8, SE-PW-3a, 
SE-PW-3b, and SE-PW-3c); item category characteristic curves of the polytomous item (SE- 
EX-9); and item information curves of the six items. We can interpret these item 
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characteristic and information curves under Rasch models to indicate that two of the 
assessment items, SE-BT-7 and SE-BT-8, are less difficult items and as such provide more 
information for low ability students than high ability students. Conversely, the item SE-PW- 
3b shows the opposite pattern. As depicted in Figure lc, this item provides more information 
about the high ability students than for the less ability students. In Figure lb, it could be 
interpreted that the students with ability less than about Theta=0.8 had high probability to get 
the item score of zero. As shown in Figure lc, the polytomous item (SE-EX-9) was able to 
grant more information than any single dichotomous item as well as provide more 
information for students with high ability. 

a) Item Characteristic Curves of the Five Dichotomous Items 
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b) Item Category Characteristic Curves of the Polytomous 
Item (SE-EX-9) 





Figure 1. Item (category) characteristic curves and item 
information curves for 2006-07 SE field test items. 



Detailed results of the model-data fit analyses for 2006-07 field testing are also 
provided in Appendix A. For example, only three items had p-values larger than 0.05 in the 
G" statistic analysis among the total 14 items in RNE 2006-07 field test forms. The other 
items appeared to misfit the Rasch models. Because it is known that the G can control the 
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type I error rates only in very limited testing conditions (Orlando & Thissen, 2000), however, 
the fit of Rasch models seemed more appropriately interpreted using the concept of INFIT 
and OUTFIT. Using this approach, in the same form every item in RNE field test forms had 
MNSQ INFIT value less than 1.5 and only three items (RN-BT-7, EX-9, and EX-6bl) 
showed some misfit problems in terms of MNSQ OUTFIT. MNSQ INFIT values provide 
model item fit information around the difficulty parameter where the item is most 
informative, the appropriate fit is very important in this area. The items in RNE field testing 
appeared to have INFIT values less than 1.24. The similar pattern appeared in the other 
domains’ 2006-07 field test forms. 

Thus, even though a few items appeared not to be a strong fit by Rasch models, this 
finding does not invalidate the measure. It simply indicates that beyond the strong overall 
achievement measured by each domain’s test forms, there are also some minor dimensions of 
achievement that impact the individual item scores of individual students. That the overall 
dimensions (or principal components) measured by each subject assessment are very strong 
is demonstrated by both (1) strong Cronbach’s alpha internal consistency reliabilities (a 
measure of measurement precision of the overall dimension derived outside the IRT model) 
and (2) the positive results from the confirmatory factor analysis and principal component 
analysis. 

POWERSOURCE® Field Test 2006-07: Experimental Comparison Findings 

As noted above, during the 2006-07 school year we field tested both the 
POWERSOURCE assessments and associated instructional materials as part of a random 
assignment study. Analyses of experimental comparisons for all POWERSOURCE units 
were completed during the current year. As noted in the previous annual report, sixty- six 6 th 
grade teachers were recruited from middle-schools in Arizona (two districts- AZ-1 and AZ-2) 
and California (two districts — CA 1 and CA 2). Within each district, teachers were randomly 
assigned to experimental (POWERSOURCE®) and comparison groups. Experimental group 
teachers in all cases participated in initial summer professional development and after school 
follow-up sessions, and used project materials-such as Checks for Understanding and 
instructional supports (including teacher instructional handbooks); whereas, comparison 
group experiences varied slightly depending on district need and configuration. District CA-1 
represented the simplest design for examining the effects of the POWERSOURCE 

(£) 

intervention in total. Here, the comparison group received no POWERSOURCE 
professional development (though teachers did participate in usual district professional 
development for mathematics) and had no access to instructional supports, although teachers 
were asked to administer the Checks for Understanding for use as a dependent variable. In 
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the other three districts, the comparison group participated in some professional development 
with their POWERS OURCE group colleagues and administered the Checks for 

Understanding , but had no access to the instructional supports, in effect providing an 
experimental test of the value added by the instructional support. All teachers gave eight 
Checks for Understanding throughout the school year — two for each of the four 
POWERSOURCE 0 modules (RNE, PA, SE, and RA). It is important to note that based on 
district curricula and needs a slightly modified module, FM, was used in the AZ districts in 
place of the RA module. Additional details about the 2006-07 field test design, materials, and 
sample can be found in Baker 2007. Preliminary results from this field test were presented in 
these progress reports based on the most complete data available at the time. We now present 
updated results from this field test, including student outcomes and an update on findings 
drawn from professional development activities. 

2006-07 field test: Student outcomes. Table 1 presents the student characteristics for 
participating schools. Hispanic students are the most dominant ethnic group across all four 
districts. For instance, the percentage of the Hispanic students in CA District 1 was 75 
percent at the time of the study. Furthermore, a high percentage of participants in the study 
were students that were economically disadvantaged, limited English proficient, and poorly 
performing in mathematics. 

Table 1 



Student characteristics for participating schools 



Student characteristics 


CA district 1 


CA district 2 


AZ districts 3&4 


Asian 


3% 


7% 


0% 


Black 


15% 


3% 


6% 


Hispanic 


75% 


38% 


64% 


Other White 


6% 


48% 


26% 


EL 


38% 


16% 


23% 


Below proficient in math, 2006 


46% 


57% 


41% 



Table 2 contains descriptive statistics for the student outcomes of interest in this 
random assignment study. Specifically, it contains the number of teachers and students; 
average score of the outcome (i.e., Checks for Understanding ) in each domain; and standard 
deviation, for the POWERSOURCE group and control group by each district. CA District 2 
had the largest number of students and teachers among the four school districts. The teacher 
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participants in the POWERSOURCE group in this district were ten for the 
POWERSOURCE group and seven or eight (depending on the unit) teachers for the control 
group. The mean scores were calculated based on the total (sum) scores of two Checks for 
Understanding (1 and 2), which have total of 13 to 19 items depending on the domain. The 
Checks for Understanding included not only short-answer items but also extended-answer 
items. 



Table 2 

Descriptive statistics of outcome by domain 







Control 






POWERSOURCE 




Domain by 
district 


n 

teachers 


n 

students 


Mean 


SD 


n 

teachers 


n 

students 


Mean 


SD 


PA a 


CA dstl 


8 


162 


7.75 


4.08 


7 


131 


12.44 


4.80 


CA dst2 


7 


139 


7.96 


4.40 


10 


210 


12.75 


4.29 


AZ dst 3&4 


11 


204 


5.30 


3.73 


13 


230 


11.42 


4.43 


Average 


9 


168 


7.00 


4.07 


10 


190 


12.20 


4.51 


RNE b 


CA dstl 


9 


161 


8.08 


3.74 


6 


109 


11.37 


4.33 


CA dst2 


8 


142 


8.27 


4.21 


10 


212 


12.13 


4.25 


AZ dst 3&4 


11 


193 


8.55 


3.71 


13 


248 


10.39 


3.84 


Average 


9 


251 


7.98 


3.93 


10 


190 


11.52 


4.23 


SE C 


CA dstl 


8 


144 


7.13 


3.09 


6 


95 


10.21 


4.00 


CA dst2 


7 


132 


7.15 


2.77 


10 


208 


9.93 


3.71 


AZ dst3&4 


11 


207 


6.33 


2.61 


13 


230 


7.98 


3.10 


Average 


9 


161 


6.87 


2.82 


10 


178 


9.37 


3.60 


RA d 


CA dstl 


3 


62 


7.71 


4.57 


3 


39 


21.23 


3.17 


CA dst2 


6 


120 


11.20 


5.93 


8 


169 


15.69 


5.99 


Average 


5 


91 


9.46 


5.25 


6 


104 


18.46 


4.58 


FM 


AZ dst 3&4 


10 


202 


6.16 


2.75 


12 


248 


7.39 


2.60 



a 17 items; 12 short; 5 extended. b 14 items; 5 short; 9 extended. 13 items; 9 short; 4 extended. d 19 items; 14 
short; 5 extended. 
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As can be seen in this table, the most noticeable finding is that all of the mean scores 
for the POWERSOURCE group in the five Checks for Understanding and four districts are 
considerably higher than those for the control group. The magnitude of the difference 
between the two groups ranges from a .45 pooled-standard deviation difference to 3.5 
pooled-standard deviation. For example, the mean score for PA in the control group across 
the four districts is 7.00, compared to 12.2 for the POWERSOURCE® group. The difference 
between these scores is approximately as large as 1.2 of the pooled standard deviation. The 
largest difference was found in CA district 1 in RA content area (approximately 13.5). 

25 

20 

c 15 

ro 



PA RNE SE RA FM 

Checks for Understanding 

Figure 2. Mean scores of Checks for Understanding for the POWERSOURCE® group and the control group in 
each district. 

Because students are nested within a teacher, two-level hierarchical modeling 

(C\ 

techniques were used to compare students’ performance between the POWERSOURCE 
group and the control group (Raudenbush & Bryk, 2002). Ignoring the nested structure of the 
data would have given rise to two major problems. First, ignoring the dependencies or 
similarities among observations (scores) for each class or teacher creates misleadingly small 
standard errors in estimating teacher effects — which creates a statistical problem for 
significance testing. Secondly, it leads to inability to detect between-teacher heterogeneity in 
the program effects. In other words, there might be variability among teachers within the 
POWERSOURCE® group depending upon fidelity of implementation, teacher 
characteristics, background characteristics of students, and many other factors that are related 
to the teacher effects. 

The following Table 3 presents the Hierarchical Linear Modeling (HLM) results using 
the POWERSOURCE® measures ( Checks for Understanding) as the outcome. The HLM 
estimate of difference corresponds to the estimate of you However, because data for different 







tl 
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CA 1 -Ctl 

■ CA1 -POWERSOURCE 
CA 2 -Ctl 

■ CA 2 -POWERSOURCE 
i AZ - Ctl 

■ AZ-POWERSOURCE 
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experiments is collected in different scales, standardizing the data is important so that the 
results are meaningful to any researcher, not just someone who is fa mi liar with a particular 
data set. A standardized effect size, 8 , is the population means difference of the two groups 
divided by the standard error of the outcome (Spybrook, Raudenbush, Liu, & Congdon, 
2006). Standardized effect sizes between 0.50 and 0.80 are considered large and effect sizes 
as small as 0.20 to 0.30 are often considered worth detecting. 



Table 3 

Estimated difference and effect size between the POWERSOURCE® 
group and the control group 



District 


n 


HLM estimate of 
diff. (p- value) 


Effect size (5) 


PA 


CA dstl 


293 


4.86 (0.004) 


1.08 


CA dst2 


349 


5.06 (0.001) 


1.12 


AZ dst 3&4 


434 


5.92 (0.000) 


1.43 


RNE 


CA dstl 


270 


3.26 (0.016) 


0.80 


CA dst2 


354 


3.84 (0.009) 


0.87 


AZ dst 3&4 


441 


1.90 (0.015) 


0.50 


SE 


CA dstl 


239 


3.33 (0.026) 


0.93 


CA dst2 


340 


2.86 (0.029) 


0.81 


AZ dst 3&4 


437 


1.69 (0.009) 


0.58 


RA 


CA dstl 


101 


13.61 (0.000) 


3.30 


CA dst2 


289 


4.81 (0.055) 


0.77 


FM 


AZ dst 3&4 


450 


1.21 (0.067) 


0.45 



Significant differences were found between POWERS OURCE and control students’ 
performance for all districts and domains. The effect sizes ranged from .45 to 3.30 across the 
four districts and the five content areas, with a median effect size of 0.84 (mean effect size is 
1.05). The median effect sizes by content area are 1.12, 0.80, 0.81, and 2.04, respectively for 
PA, RNE, solving equations, and review and applications. The median effect sizes by district 
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ranged from 0.54 to 1.01. These findings pertain to POWERSOURCE® tasks, which are 
broadly representative of fundamental pre-algebra content. 




PA RNC SC RA FM 
Checks for Understanding 



■ CA 1 

■ CA2 

■ AZ 



Figure 3. Estimated effect sizes for each Checks for Understanding and each district. 



In terms of next steps, we are in the process of collecting and analyzing both prior and 
concurrent state standards test data (i.e., CST in CA, AIMS in AZ) in mathematics for both 
the POWERS OURCE and control groups. The prior year’s data would serve to document 
the initial comparability of the two groups and could be used as a covariate for analysis; the 
concurrent year data would serve as a partial transfer measure. However, we are concerned 
about the sensitivity of the state test measures to the intervention, in that the state tests cover 
a much broader domain than the four content areas addressed by POWERSOURCE . We 
also are attempting to obtain item level data so that we can examine performance on the 
items related to POWERSOURCE content, but even given the availability of this data, the 
reliability would need to be determined. 

2006-07 Field Test: Professional development and teacher outcomes. As described 
in earlier progress reporting (Baker, 2007), the design of the professional development 
component of POWERSOURCE is based on findings in the field of cognitive science about 
how students leam, on expert-novice literature that suggests how expertise in a subject like 
mathematics develops, on the role of formative assessment in facilitating this process, and 
how these components can be effectively combined to improve teacher practice. 

As also detailed in our previous progress report, as part of the 2006-07 field test we 
examined changes in teacher knowledge across professional development participation based 
on an activity that asked teachers to rank a series of student assessment responses (Baker, 
2007). This task requires teachers to rank five students’ responses in order of difficulty. It 
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consists of two sets of items, the first intending to measure how well students can describe 
the algebra principle, while the second intended to measure students’ knowledge of algebra 
principles. This activity was integrated into the POWERSOURCE professional 
development meetings at both the beginning and end of the 2006-07 school year. We 
analyzed results from this measure both to provide some basic statistical quality information 
about the measures and to assess pre- and post- professional development differences in 
teacher knowledge. 

Based on teacher responses to a background questionnaire, 43% of the participating 
teachers reported attaining a masters degree or higher, with the remaining 57% holding a 
Bachelor’s degree. Prior years’ teaching experience as reported by teachers, both in general 
and specific to mathematics, was quite varied with a mean of 9.2 and 7.7 for general and 
mathematics teaching experience, respectively. Table 4 summarizes the results of teacher 
responses related to self-evaluation of their algebra background knowledge. Overall 
approximately 50% to 70% of teachers felt they had high to very high relative knowledge; 
respondents were least confident about the distributive property. 



Table 4 

Relative Knowledge in Specific Mathematics Knowledge Domains 

Percent of respondents 



Relative level of knowledge 


Low 


Average 


High 


Very high 


RNE 


2.8 


26.4 


47.2 


23.6 


Distributive property 


2.8 


31.9 


38.9 


26.4 


Principles for solving equations 


2.8 


27.8 


45.8 


23.6 



In general, there was relatively little correlation between teacher experience and their 
perceptions of relative knowledge (r <0.25). Teachers were fairly consistent, however, with 
inter-correlations of the knowledge self-evaluation domains in Table 4 (r = .80). 

A total of 37 teachers who participated in professional development activities offered 
during the 2006-07 school year completed the post-assessment 1 . The completion rate was 
lowered by one participating district’s request that their teachers did not complete this 
follow-up knowledge measure. 

1 As noted earlier in this report, in some of the participating districts both POWERSOURCE© (i.e., using the 
POWERSOURCE© instructional materials) and control teachers (did not use POWERSOURCE© instructional 
materials) attended development activities. 
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In relation to task reliability, the two ranking tasks formed fairly consistent scales in 
terms of the response patterns. That is, responses formed a scale wherein the easiest item that 
was missed consistently related to subsequent more difficult items being answered correctly. 
For example, in set 1, if the easiest item was missed, it was extremely unlikely that any other 
item would have been ranked correctly. In other words, of those teachers who correctly 
answered the second most difficult item, answered the least difficult item correctly 98.3% of 
the time. The reliability of each of the tasks was evaluated using Cronbach’s alpha. The 
reliability for set 1 is approximately, a 0.84. The reliability of set 2 was considerably lower 
(a = .40), and this low reliability would translate into low reliability for assessing change. 
Hence, set 1 is examined independently for change as well as a combined, equally weighted 
composite of set 1 and set 2. The setl/2 composite reliability (a = .73) is acceptable for 
examining pre-post change. 

Before we evaluate changes in teacher knowledge, we briefly examine whether those 
teachers who have both pre- and post- scores are representative of all teachers with pre- 
scores. Two substantive differences arise: 1) teachers with higher setl/2 pre- composite 
scores tended not to complete the post-assessment, and 2) more teachers without advanced 
degrees (BA only) completed the post-assessment. Specifically, the average pre-test 
composite score among pre-test only teachers for setl/2 is 7.1, while it is 5.8 for teachers 
completing both the pre- and the post- test. Hence it is not tenable to assume that the teachers 
taking both assessments are representative of those taking the pre-test only; thus, results 
pertaining to change need to be interpreted with caution. 

Given these caveats, we then evaluated changes in teacher knowledge for that subset of 
teachers that have both pre- and post- test scores. We examined change in teacher knowledge 
by utilizing a two level random effects model that models teacher ranking scores at level 1 
and teachers at level 2. Similar to Bryk, Thum, Easton, and Luppescu (1998) we re- 
parameterized the level one model in order to estimate true initial status and gain. 

Table 5 summarizes the results for set 1 rankings. The results indicate that among 
teachers who participated in both the pre- and post- assessments there was no systematic 
change in scores. 
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Table 5 



Random Effects Model for Teacher Knowledge Set 1 



Status 


Estimate 


SE 


Approx /?- value 


Initial status 


4.00 


0.21 


<.01 


Gain 


0.05 


0.27 


0.862 



We next examine changes in the set 1/2 composite. The results are displayed in Table 

6 . 



Table 6 

Random Effects Model for Teacher Knowledge Set 1/2 



Effects 


Estimate 


Variable 


Approx p- value 


Fixed effects 




SE 




Initial status 


4.1 


0.22 


<.01 


Gain 


1.2 


0.22 


<.01 


Random effects 




s.d. 




Initial status 




1.10 


<.01 


Gain 




0.91 


<.01 



The results in Table 6 indicate that teachers gained about 1.2 scale points (an effect size 
of about 0.8) pre- to post-. Further, the results indicate that both initial status and gain vary 
among teachers. Hence, we expanded the unconditional model to include teacher information 
that might account for pre-existing differences in initial status as well as factors related to 
growth (Osgood & Smith, 1995). 
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Table 7 



Random Effects Model for Teacher Knowledge Set 1/2 



Effects 




Variables 


Estimate 


SE 


Approx p 


Fixed effects 








Initial status 


4.08 


0.20 


0.00 


Yrs. teach math 


-0.05 


0.04 


0.23 


Knowledge: RNE 


-0.41 


0.43 


0.35 


Knowledge: : Distr. 


0.30 


0.35 


0.39 


Knowledge: : Solve eq. 


0.10 


0.34 


0.78 


MA 


1.09 


0.50 


0.04 


Gain 


1.22 


0.17 


0.00 


Yrs. Teach math 


0.07 


0.03 


0.06 


Knowledge: RNE 


0.76 


0.37 


0.05 


Knowledge: : Distr. 


-0.19 


0.45 


0.68 


Knowledge: : Solve eq. 


0.18 


0.49 


0.72 


MA 


-0.54 


0.48 


0.27 


Random effects 


SD 


Approx p 


Var. reduction 


Initial status 


0.97 


0.02 


19.8% 


Gain 


0.46 


0 


44.5% 



The full model results presented in Table 7 indicate that only MA is significantly 
related to teachers’ pre-test knowledge. Consistent with expectations, teachers with MA 
degrees score about 1 point higher that teachers with BA degrees. None of the self-perceived 
knowledge levels are related to teacher pre-assessment results. 

There is some suggestive evidence (p < .10) that teachers with more experience 
teaching mathematics gained slightly more in teacher outcomes pre- to post-. That is 
comparing a teacher who is one standard deviation above average in experience with one that 
is one standard deviation below average in mathematics teaching experience, the more 
experienced teacher is expected to gain approximately 1.3 points more than the lest 
experienced teacher. 

The results also indicate that self-perceived RNE knowledge is related to larger teacher 
knowledge gains. Comparing a teacher who reports low RNE knowledge to one that reports 
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very high knowledge, the teacher reporting a very high RNE knowledge level would be 
expected to gain about 2.3 points more than the teaching reporting a low level. 



Overall, the full model is able to account for about 20% of the variation in pre-test 
results and about 45% of the variation in true gains. 




Pre - Post Change 



YRS_MATH: lower 
YRS_MATH: mid 50% 
YRS_MATH: upper 



Figure 4. Effect of mathematics teaching experience. 



Figure 4 highlights several points; for instance, it depicts the variability in pre-test 
scores, the variability of gains, and the effect of teacher experience teaching mathematics. It 
should be noted that the results presented in Figure 4 are re-scaled composite scores for set 
1 / 2 . 

In summary, we found the teacher knowledge ranking task to be a short, relatively non- 
invasive tool for assessing teacher knowledge; further, it produces relatively reliable results 
as well as a scale that allows for cumulative interpretations. That is, the pattern of correct 
responses is very consistent as items move from easiest to most difficult. For this reason, 
scores are more than simply an index of a teacher being able to answer any three of five 
questions correctly; rather, a score of three indicates which three questions a teacher likely 
completed successfully. This provides valuable, as well as efficient, results from which 
mentors or trainers can identify the areas in which teachers are having difficulty. 
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Among teachers who took both the pre- and the post- Ranking assessment, the 
composite set 1/2 scores demonstrate statistically significant and substantively important 
gains. Moreover, gains are related to teachers’ mathematics teaching experience as well as 
the self reported RNE knowledge. However, teachers demonstrate a fair amount of variability 
in pre-test scores that is not accounted for with the teacher information available in the data; 
this area certainly warrants further study. 

It is important to note that, due to practical constraints, the gain results are based on a 
subset of teachers who took the pretest and that this subset is not representative of the entire 
sample. In fact, it appears as if those teachers with the lowest composite scores were exactly 
those that took the post-assessment, which highlights potential regression to the mean 
concerns. However, these results provide some initial support for the effectiveness of the 
professional development activities developed as part of POWERSOURCE®, as well as 
emphasize the need for additional systematic data regarding teacher outcomes of 
POWER S O URGE participation. As described later in this report, the 2007-08 project 
activities included additional teacher outcome measures for both POWERSOURCE and 
control teachers. 



POWERSOURCE® Implementation Study 2007-08 

The core undertaking of our work during the 2007-08 school year was conducting an 
extended, random assignment implementation study of our 6 th grade POWERSOURCE® 
program. As with the 2006-07 field test, teachers were randomly assigned to either 
POWERSOURCE or control conditions with the ultimate goal of determining program 
impact on both students and teacher learning outcomes. The 2007-08 study differed from the 
previous year’s work in a number of important ways, however. Specifically: 

• The experimental design incorporated both within- and between-school random 
assignment models. That is, for some of the districts the random assignment 
accomplished within each school (i.e., a given school had both 
POWERSOURCE and control teachers), and for some the random assignment 
was between school (i.e., all teachers at a given school were POWERSOURCE 
or control). Additional background and rationale for this approach can be found in 
the CRESST supplement design report submitted to IES in August, 2007. 
Ultimately, a total of 112 6 th grade teachers across seven school district 
participated in the study. 

(S) 

• Although the content focus of the four POWERSOURCE modules remained the 
same (RNE, PA, SE, and RA), based on teacher feedback and our implementation 
experiences in 2006-07 the structure of each unit changed somewhat. In 2007-08, 
POWERSOURCE teachers were provided with three Checks for Understanding 
for each unit - one prior to the first day’s set of instructional materials, one in 
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between the first and second day of instruction, and one after the second day of 
instruction. Thus, the students completed 12 Checks for Understanding (three for 
each of the four units) during the school year. The Checks for Understanding and 
instructional materials were also revised/refined based on analysis of 2006-07 
item level data. 

• Unlike 2006-07, the control students did not complete any of the Checks for 
Understanding (i.e., the short formative assessments). Thus, the 2007-08 control 
students and teachers had no exposure to any of the POWERS OURCE materials 
or concepts during the school year. 

• All students (POWERS OURCE and control) completed a test of prerequisite 
knowledge at the beginning of the school year and a transfer measures of math 
knowledge at the end of the school year. The test of prerequisite knowledge 
serves as a baseline measure for later analyses, while the transfer measure will 
serve as an independent, student outcome measure (in addition to state test data). 

• Based on district response and feedback, districts were offered the option of the 
control teachers receiving an alternative (i.e., non-POWERS OURCE ) 
professional development from CRESST (as opposed to the control teachers not 
receiving any additional professional development than what the district already 
had planned). The majority of participating districts selected this option. 

Below, we summarize changes made for the treatment and comparison conditions for 
the 2007-08 implementation study (including the alternative professional development 
offered to the control teachers), followed by brief descriptions of the design, measures and 
analysis plan for the study. Additional details about the plan and its rationale can be found in 
the supplemental design report submitted to IES in August, 2007. The data collection for 
these activities is in its final stages. 

Revisions to the Treatment and Control Conditions for the 2007-08 Study 

Revision of 6 th grade Checks for Understanding and instructional materials. The 

Checks for Understanding were revised based on data from the 2006-07 study, whose 
analyses were described in the earlier sections of this report as well as in Appendix A. 
Moreover, three Checks for Understanding were developed for each of the four 6 th grade 
POWERS OURCE 0 domains, — instead of the two used the previous year, there were 
ultimately three Checks for Understanding per unit. The initial Check for Understanding 
consisted of between 8-10 items and was given prior to instruction in the relevant content. 
This Check for Understanding acted as a baseline assessment for each student and a) gave 
teachers information about their students initial knowledge and b) allowed us to compare 
students’ performance before and after instruction, thus, providing information on the 
instructional sensitivity of the Checks for Understanding. Each subsequent Check for 
Understanding (of which there are two) consisted of 4-5 items (2 symbolic 
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representation/computation items and 2-3 open-ended problem solving and/or explanation 
tasks). Based on our research and development over the last two years, teachers’ procedure 
for using the Checks for Understanding was as follows: 

1. Administer an initial Check for Understanding big idea and its applications (15-20 
minutes) and analyze results. 

2. Present instructional activities (if necessary) addressing deficiencies in conceptual 
understanding identified in step 1 (one class period). 

3. Administer a second Check for Understanding focusing on conceptual 
understanding (15 minutes); follow up instruction if necessary. 

4. Present instruction on applications of the big idea to problem solving and symbolic 
representation and computation tasks (if necessary) (15 minutes). 

5. Administer a third Check for Understanding focusing on conceptual understanding 
(15 minutes); follow up instruction if necessary. 

Please see Appendix B for an example of a teacher handbook containing the revised 6 th 
grade Checks for Understanding and instructional materials. 

Revision/addition of professional development resources for 2007-08. While the 
overall program and structure of POWERSOURCE professional development during the 
2007 -08 school year was very similar to the program of professional development delivered 
in the 2006-07 school year, there were notable additions to enhance the strength of the study 
and to address the concerns and needs of the district partners in terms of the experiences of 
the control teachers. 

As noted above, our participating districts insisted that if they were to participate in a 
random assignment study then their control teachers must receive some sort of professional 
development through their participation. The addition of such professional development adds 
strength to the study design, as well as limits some confounding of experimental and control 

(G) 

group treatment in the prior year’s study. Thus, in addition to the POWERSOURCE 
professional development, we created two separate alternative (control) programs of 
professional development that control teachers participated in during the school year (two 
different programs were developed to meet the differing needs/agendas of the participating 
districts). The structure and length of these alternative professional development sessions - 
an initial one day institute plus follow-up after school meetings — was identical to that of the 
POWERSOURCE® professional development. 

We created the first of the two alternative professional development programs for 
districts that use the Data Director™ assessment data management system. Control teachers 
in these districts received an identical amount of professional development as the 
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POWERSOURCE teachers, but this professional development concerned information and 
reports provided by Data Director™. Specifically, this professional development focused on 
providing a method for users to analyze the technical quality of district benchmark 
assessment forms and items. While the focus of these sessions was on assessment, the 
researchers provided no actual guidance on how to use the information from the assessments 
or guidance on how to create instructional plans. 

The second complete alternative program of professional development was developed 
for use in districts where Data Director™ was not available. It is an educational psychology- 
based, four unit program centered on motivational factors in the classroom and their role in 
student learning. It introduces teachers to research-based motivational theories and highlights 
their connections to everyday classroom practices. The units each emphasize a specific area 
of research and offer some practical ways to incorporate the research findings into daily 
practice. Issues covered included personal responsibility, self-efficacy, attributions and 
affect, and teacher expectations. 

Although these alternative programs of professional development were quite different 
from one another and from the POWERSOURCE professional development, they were 
designed to be credible, stand-alone offerings based on current educational theory and 
research. Sample activities from both of these alternative sessions are presented in Appendix 
C. 

As with the 2006-07 pilot test, the POWERSOURCE 0 professional development for 
2007-08 aimed to provide an intellectually stimulating and supportive environment, that 
builds teacher knowledge and pedagogical content capacity, provides time for reflection, and 
monitors the effectiveness and the impact of our activities. 

Teachers were asked to evaluate each professional development meeting they attended 
using a satisfaction survey. This survey asked teachers to rate the organization of the 
professional development, presenter, content, benefit to the teacher, and their overall 
impression on a 4- point scale. The teachers were also asked short-answer response questions 
concerning the helpfulness of the content presented, and the percent of new content contained 
within the presentation. Finally, the survey solicited any additional comments teachers may 
have about the professional development meeting. These surveys were filled out by each 
teacher at the end of each professional development meeting and were collected before 
teachers departed. 

The data from the 2006-07 POWERSOURCE 0 teacher satisfaction surveys were highly 
positive and we expect this year’s results to reveal similar findings. Specifically, 77% of the 
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teachers found the meeting presentation to be “excellent” while another 20% rated the 
session as “good”. When rating the session presenter, 76% found the presenter to be 
“excellent” and 21% found the presenter “good.” On the content presented, 63% found the 
content “excellent” and 32% found it “good.” Sixty percent of the teachers rated the benefit 
of the professional development to themselves as an educator as “excellent,” while 31% 
found it to be “good.” Overall, 67% of teachers rated their overall experience at 
POWER S O URGE meetings to be “excellent” and 30% rated it as “good.” Teacher open- 
ended remarks exemplified their happiness with the professional development meetings. As 
one teacher stated, “These sessions are the best professional development I have experienced 
during my time as a teacher. The presenters help me think outside the box and get creative 
about how to bring understanding to students.” Another teacher commented that, “I have 
never taught this [content] so completely. This will help me as much as the students.” 

In order to supplement the face-to-face professional development, researchers also 
created a password protected professional development website with additional resources. 
Each of the four major sections of the website addresses one or more of the big ideas that are 
the focus of POWERS OURCE . Every section, in turn, discusses why a particular big idea 
or its integration is important; how that big idea appears across the 6 th grade curriculum and 
will be used in future grades; misconceptions that students often hold about that idea; and 
frequently asked questions that teachers have about teaching the idea. It is intended that this 
website will serve two primary purposes. First, it will provide teachers in the treatment group 
with an additional site to acquire the information that we present at professional development 
sessions. In other words, teachers can visit the site and repeatedly review the material they 
heard at professional development. Secondly, we intend to use the website to help us build 
capacity in the participating districts themselves. Eventually, we will allow all teachers 
(treatment and control) in participating districts to gain access to our POWERSOURCE 0 
materials. The website will allow these teachers and districts to access and download the 
POWERSOURCE materials. In this way, these materials can form a core for ongoing 
professional development in the future. Sample materials from the professional development 
website can be found in Appendix D. Our project professional development experiences over 
the last three years have provided some important lessons that we learned and case studies of 
how formative assessment contributes to teacher development and student learning. 

Overview of 2007-08 Implementation Study Design/Sample 

As described in previous progress reports (Baker, 2007) the project had to make some 
sample adaptations due to the decision of Los Angeles Unified School District not to 
continue in the implementation (i.e., random assignment) study beyond the 2006-07 school 
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year. CRESST spent considerable time and effort to secure additional school district 
participation in the project for the 2007-08 school year — meeting with representatives of 
close to 20 districts — and continues to do so in an ongoing effort to strengthen and expand 
the sample. Ultimately, seven small-to-medium sized school districts participated in the 
random assignment 2007-08 implementation study and all but one are in California. As 
compared to the larger districts that we spoke with, districts of this size seemed to have fewer 
algebra initiatives already in progress; they also appeared to be more open to participating in 
an experimental study that would result in not having all teachers in the district using the 
same program and materials. 

Although the inclusion of a larger number of districts than originally planned was 
driven by practical considerations, methodologically it is also a definite strength. It will 
provide us with an opportunity to see how well the program works in a variety of settings — 
both in terms of analysis of quantitative data and more qualitative/descriptive indicators. 
Furthermore, it will be possible to investigate factors that might magnify or dampen the 
effects of the program. This facet of the study adds the qualities of a rich, mixed methods 
multi-site case study. With one large district, we might get a very precise estimate of the 
treatment, but it would pertain to only one set of district conditions. 

As noted earlier, overall there were 112 (63 POWERSOURCE® and 49 control) 
teachers across seven school districts (totaling 28 schools) participating in the 2007-08 study. 
As also described previously, we used two designs — within- school and between- school, 
which was based on district needs and configuration. In districts with strong grade level team 
collaborative initiatives, for instance, a within-school design would not be tenable (as both 
treatment and control teachers would be regularly collaborating on curricular issues, thereby 
potentially exposing the control teachers to POWERSOURCE® concepts and materials). 
Ultimately, three of the districts used a within-school design, where random assignment was 
accomplished within each school (i.e., a given school had both treatment and control 
teachers). Four districts used a between- school design, where schools within a district were 
randomly assigned to treatment or control conditions. The total number of students in the 
2007-08 POWERSOURCE® and control samples are approximately 3,600 and 2,900, 
respectively (although post- measure data collection is still underway). 

Special Measures for the 2007-08 Study 

In addition to revisions to the POWERSOURCE® materials for the 2007-08 
implementation, there were also some additions and refinements to the project outcome 
measures. These are described below. 
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Student transfer measure. While we were pleased with the very strong effect sizes 
from the 2006-2007 study based on POWERSOURCE 0 measures, we recognized the need 
for demonstrating intervention effects on an independent, transfer measure. We hypothesized 
that students in the POWERSOURCE group would possess a better understanding of the 
basic mathematical principles contained within each domain. We also hypothesized that 
students would be able to apply concepts they have learned, solve complex problems and 
transfer the principles covered by the POWERS OEiRCE domains outside of the specific 
program materials. For example, having received instruction and formative assessment on 
RNE, students should understand the multiplicative identity principle and be able to use it to: 
a) demonstrate that a set of rational numbers are equivalent, b) find equivalent fractions, c) 
find missing numbers in proportions and d) solve proportional reasoning problems. In order 
to answer these questions we developed a transfer measure to compare the 
POWERS OURCE and control groups on novel items related to our four 

POWERSOURCE domains. This instrument will provide an additional student outcome 
measure to compliment results from student state standardized tests (which will be provided 
to CRESST by the participating school districts). 

The transfer measure was developed using items from several sources including Trends 
in International Mathematics and Science Study (TIMSS); the National Assessment of 
Educational Progress (NAEP); the Qualifications and Curriculum Authority (QCA) Key 
Stage 3 exam; the Programme for International Student Assessment (PISA); and benchmark 
tests used in one of our pilot districts (see Appendix E for additional item source 
information). An initial set of 44 items were selected from the various sources. Items were 
selected based on their relevance to the POWERSOURCE domains and their 
appropriateness for a transfer task (i.e., related to POWERSOURCE content but not exact 
replicas of item types used in the Checks for Understanding). A final set of items (29) were 
selected from the initial 44 items. Of these items 19 were multiple choice, 9 short answer and 
one explanation task. Items were selected based on their representation in the California state 
standards and relevance to POWERSOURCE items (see Appendix F for alignment of items 
to California standards and the National Council of Teachers of Mathematics (NCTM) Focal 
points). Some of the initially developed items were deemed more appropriate for 7 th grade 
and will be used for the 7 th grade transfer measure. 

The transfer measure was given to all participating students (POWERSOURCE 0 and 
control) at the culmination of the 2007-08 school year. Data are still being collected and will 
be analyzed beginning in the summer 2008. 
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Teacher outcome measures. As described in the August 2007 design supplement, 
several teacher outcome measures were included in the 2007-08 implementation study. These 
included: 

• Knowledge map task: The knowledge map task asks teachers to complete a 
graphical network of key concepts in algebra, including their links and 
interconnections. These maps are scored in comparison to expert maps of the same 
concepts. All teachers (POWERSOURCE and control) completed this task twice, 
at the beginning and end of the school year. 

• Teacher pre/post survey: All teachers (POWER S O URGE and control) completed a 
survey measure at the beginning and end of the school year. This measure asked 
teachers about their expertise in a variety of mathematics areas, their instructional 
attitudes/beliefs, as well as asked them to evaluate samples of student work. The pre 
measure also included background items about teachers’ prior math experience and 
training. These measures are included in Appendix G. 

• Teacher implementation surveys: All teachers completed a series of short surveys 
during the school year about their classroom math practices and assessment use. 
These surveys were timed to coincide with the POWERSOURCE modules (e.g., 
RNE, PA, SE). The POWERSOURCE® teachers’ implementation surveys included 
additional questions about their POWERSOURCE materials use and feedback. 
The surveys are included in Appendix G. 

Observation/interview study. As part of the 2007-08 POWERSOURCE® 
implementation study we conducted a pilot study of a classroom observation and teacher 
interview measure. The purpose of this study was twofold: 1) to investigate the relationship 
between teacher self-report (survey) measures and actual practice and, 2) to provide more 
detailed information and feedback about the POWERSOURCE materials and how they are 
being used in the classroom. A total of 12 teachers participated in this qualitative study and 
they were selected to represent a range of districts and prior teaching experiences. All 
participating teachers were observed implementing the RA instructional materials in their 
classroom; moreover, they participated in an in-depth, semi- structured interview. The RA 
unit was selected for the observations as it represents an integration of all of the 
POWERSOURCE® units across the school year. The observation protocol attempts to obtain 
information about how teachers actually use formative information, how concepts are 
presented (e.g. algorithmically, conceptually or both), and how the big ideas are integrated 
with instruction on other concepts in the 6 th grade math curriculum. The interview protocol 
focused on information about the teachers’ thinking on the use and integration of formative 
assessment information and their rationale about how to incorporate the big ideas into their 
instructional planning and teaching. In addition, the interview was designed to solicit teacher 
feedback about the POWERSOURCE materials and professional development. The 
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interviews and observations were conducted by pairs of researchers. Prior to data collection, 
the researchers participated in one day training about use of the interview measures, and 
reached concordance on observation ratings. The interview and observation protocol are in 
Appendix H. 

Analysis Plan for 2007-08 Implementation Study 

We have adapted our original analysis plan given two different research designs and 
samples described in the previous section. Specifically, there are two changes that affect the 
original analysis plan. First, we included schools from multiple districts in the sample (seven 
different school districts). Due to this change, we need to consider variability in the effects of 
POWERS OURCE across districts. Second, there are two different research designs mixed 
depending upon district idiosyncrasies. In other words, there are one groups of 
schools/districts in which only between school (B-S) design should be employed, while there 
also the other groups of school/districts where only within-school (W-S) design (addition 
details about design and analysis considerations can be found in the design supplement 
submitted to IES in August 2007). In the following section, we will describe our current 
plans/considerations for analyses of the 2007-08 implementation study data — with the 
understanding that some aspects of the analyses will be refined depending on the 
characteristics of the data once it has all been collected. 

Within-schools (W-S) and between-school (B-S) design considerations. An 

advantage of the W-S design is that each participating school provides us with a mini- 
experiment, that is, each school provides us with an estimate of the effects of 
POWERS OURCE . Drawing on the implementation data and other background information 
that is available, this will provide some possibilities to generate insight into why the program 
seemed to work very well in some schools but not as well in others. 

Another advantage of W-S designs is that they can be fairly powerful. In the case of B- 
S designs, a key source of variability that shows up in the standard error of the estimate of 
the treatment effect is the variability between treatment schools in their post-test scores and 
the variability between comparison schools in their post-test scores. In a district where, for 
example, the intake characteristics considerably differ from school to school, this source of 
variability could be substantial. In a very large urban school district, the likelihood of 
variability across schools in terms of their intake characteristics is very high. But if intake 
characteristics and other school characteristics do not differ a great deal (which might be the 
case of a small suburban school district), this source of variability is likely to be fairly small. 
In the case of W-S designs, the variability between schools in their post-test scores does not 
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enter in the standard error of the estimate of the average treatment effect of the sample of 
schools; hence, this is a large advantage in terms of precision. However, the variance in the 
effects of the treatment across schools matters in the standard error of estimate. All things 
being equal, the more the effect of treatment varies across the schools, the larger the standard 
error. 

By district analyses. The first analysis option is to employ a three-level hierarchical 
model in each district. District-by-district analysis yields a fairly precise estimate of the 
treatment. In certain smaller districts the precision of the estimate of the treatment may be 
lowered due to the small number of teachers/schools, but this may be tempered a bit by the 
fact that in these districts, schools generally do not differ widely from each other in terms of 
student and school background characteristics. We will also consider some pooling of 
information in these analyses where it seems sensible. For example, a pooled estimate of 
between-school variance could be computed using the data from two small districts 
employing between-school design. Note that this kind of pooling would be similar to what 
programs like HLM (Raudenbush, Bryk, Cheong & Congdon, 2004) do regularly when 
estimating variance components. In districts where a W-S design is employed, again 
precision will likely be lower in the case of smaller districts. As with the B-S design, the 
pooling technique will also be applied as appropriate. 

In sum, how precisely a district’s treatment effect is estimated will depend on a variety 
of factors: where a W-S or B-S design is employed in the district; the number of schools, 
teachers, and classes in a district that participate; and the magnitude of the various sources of 
variance discussed. 

Combining information across the districts. One challenge is how to proceed in 
combining this information to estimate an “Intent-to-Treat (ITT)” effect. One strategy would 
be to employ two-stage meta-analytic technique. First, we analyze data for each district 
separately and obtain an estimate of the POWERSOURCE effect for each district and its 
standard error. As mentioned previously, we might need to do a little pooling of information 
to estimate certain variance components. Then the district estimates of the effects of 
POWERSOURCE® and their standard errors could be read into HLM program or WinBUGS 
(Spiegelhalter, Thomas, Best, & Lunn, 2003) and synthesized, as in a meta-analysis. This 
would give us the estimate of the ITT effect, its standard error, and, importantly, an estimate 
of the variance in the effects of POWERSOURCE across districts. HLM will also provide 
us with a test of the heterogeneity in treatment effects across districts. With only seven 
districts, the estimate of the between-district variance in treatment effects would not be very 
precise, but still it will provide useful information about its magnitude. Note that an 
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advantage of using WinBUGS to do this analysis is that it will give us a posterior distribution 
for the between-district variance in treatment effects, which will give us a sense of how 
probable it is (given the data) that this parameter exceeds values that are deemed 
substantively meaningful.) 

Note that there are some precedents in the literature for doing these sorts of two-stage 
analyses. A classic example is Rubin’s synthesis of the results from eight parallel randomized 
experiments of the effects of coaching for the Scholastic Achievement Test (SAT; Rubin, 
1980). In each of eight schools, kids were randomly assigned to coaching or to a control 
condition. The data were analyzed separately from school to school in order to obtain an 
estimate of the coaching effect for each school and its standard error. Rubin then synthesized 
these eight estimates in an analysis much like the WinBUGS analysis mentioned previously. 
For other examples in the literature, see Raudenbush & Bryk, 1985, 2002. 



3-Level Hierarchical Model for Within-School Design 
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3-Level Hierarchical Model for Between-School Design 
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Development and Pilot Testing of 7 th Grade Materials 

In addition to the implementation study at 6 th grade, during the 2007-08 school year we 
developed and pilot tested materials for 7 th grade POWER S O URGE modules, towards the 
goal of adding 7 th grade teachers to the experimental study in 2008-09. This work included 
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the development and testing of 7 th grade Checks for Understanding as well as the 
development of instructional and professional development support materials. 

Development of 7 th Grade Items 

During the 2007-08 year, we pilot tested a set of new 7 th grade assessment items. 
Working with expert math teachers, and using data from the 6 th grade Checks for 
Understanding we developed a set of 7 th grade items reflecting the three conceptual 
POWERS OURCE domains culled from our big ideas list — rational number equivalence, 
principles for solving equations, the distributive property and applications of these big ideas 
in other critical areas of mathematics. Items were reviewed by math experts as well as a 
group of five experienced middle-school math teachers. 

The three conceptual domains chosen for inclusion in the POWERS OURCE study 
were selected because they are: a) heavily represented in state standards and state and district 
test blueprints; b) historically difficult for students to master; and c) important prerequisites 
for learning and mastering algebra. Given that the conceptual domains for 
POWERS OURCE remain constant across the grade levels, we paid close attention to how 
the concepts develop across grades. Thus, in developing the 7 th grade assessment items, we 
looked at the learning trajectory for each domain. In the 6 th grade, students were learning 
how to solve one-step linear equations; whereas, in 7 th grade they begin to solve more 
complex, two-step equations. The big ideas underlying the concepts remain the same; yet, the 
skills students are mastering are different and build on what was previously learned. Our 
POWERSOURCE materials and Checks for Understanding assessments reflect this 
trajectory and make the necessary connections between 6 th and 7 th grade content and the big 
ideas (see Table 8 for an example of the California state standards that relate to solving 
equations). 
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Table 8 

CA Standards relating to Principles for Solving Equations (6 th -8 th grades) 



6 th Grade 



7 th Grade 



8 th Grade 



1.1 Write and solve one-step 
linear equations in one variable. 

1 .2 Write and evaluate an 
algebraic expression for a given 
situation, using up to three 
variables. 



4. 1 Solve two-step linear 
equations and inequalities in one 
variable over the rational 
numbers, interpret the solution or 
solutions in the context from 
which they arose, and verify the 
reasonableness of the results. 

1.1 Use variables and appropriate 
operations to write an expression, 
an equation, an inequality, or a 
system of equations or 
inequalities that represents a 
verbal description (e.g., three less 
than a number, half as large as 
area A). 



4.0 Students simplify expressions 
before solving linear equations 
and inequalities in one variable, 
such as 3(2x-5) + 4(x-2) = 12. 

5.0 Students solve multi-step 
problems, including word 
problems, involving linear 
equations and linear inequalities 
in one variable and provide 
justification for each step. 



Pilot Testing of 7 th Grade Items 

Over 150 7 th grade items have been developed and 72 items pilot tested in 27 
classrooms with nine teachers in four schools. Using the same assessment model as the 6 th 
grade items, we have developed six types of assessment: basic computation tasks, partially 
worked problems, explanation tasks, word problems and problems involving graphics. Items 
were grouped together (within domains) to create the Checks for Understanding assessment 
forms. We used an overlapping design to allow us to compile item data and conduct IRT 
analyses on all items. The 72 items that we have pilot tested to date were compiled into 46 
forms (17 forms for solving equations, 19 forms for PA, and 10 forms for RNE). 

Pilot testing process. For pilot testing, the tasks described were assembled into forms 
that students should be able to complete in about 15 minutes. This time frame was imposed 
by the districts with whom we collaborated. Any assessment that was longer, they felt, would 
be viewed by teachers as a test, and would evoke complaints about too much district testing. 
However as it has turned out, this time frame actually has a number of advantages; for 
instance, it focused teachers and students’ attention on students’ understanding of a single 
concept as well as encouraged deep assessment without being too intrusive into or 
engendering teacher hostility about intrusion into instructional time. 

Each teacher participating in pilot tests received at least two different test forms, each 
focusing on the same big idea, with each form containing between 3-5 tasks. The forms were 
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randomly assigned to students within classrooms; each teacher administered the assessments 
to all of their 7 th grade students. In all cases the first two to three items on the test forms were 
basic computation items. The subsequent items were either open-ended explanation tasks, 
partially worked problems, word problems, or problems with a graphic prompt. Forms 
containing explanation tasks did not contain any other tasks besides the basic computational 
items. 

All pilot data from the closed-ended responses were entered by a group of 
undergraduate and graduate student workers and other CRESST staff. Three-point scoring 
rubrics were developed for the open-ended items. Training was conducted for scoring session 
participants. 

Selecting items for inclusion in the 7 th grade Checks for Understanding. From the 
set of 7 th grade items piloted in the 2007-2008 year, we will choose items to include on our 
Checks for Understanding forms and instructional materials for the extension of the 
POWER S O URGE study in 7 th grade (to be conducted in the 2008-2009 school year). Data 
from the pilot test are currently being analyzed and we will use the same procedures for 
analyzing data and selecting items as we used for the 6 th grade Checks for Understanding. 
That is, as indicated earlier, we will employ several criteria to evaluate the items used in the 
pilot-testing phase. These include: confirmatory factor analyses, reliability analyses, and IRT 
analyses. Specifically, our typical analysis scheme for each extended set of Checks for 
Understanding from each pilot test form was to first calculate reliability coefficients 
(Cronbach’s alpha) for items representing each domain. Second, as another check of item 
quality, we conducted a principal component analysis and a confirmatory factor analysis for 
each test form to check whether the items exhibited the factor structure we expected (e. g., 
whether the computation items loaded on the same factor, etc). Third, IRT analyses based on 
Rasch models were conducted in order to obtain item parameters (difficulties) and item 
characteristic and information curves so that we could use them to select items for future 
testing. The model-data fit was investigated using two model fit indices. One is the G2 index, 
which is the Chi-square (%2) statistic and provided in PARSCALE phase 2 outputs, and the 
other is the MNSQ statistics. 

7 th grade instructional materials and professional development design. Concurrent 
to the development of the Checks for Understanding items in 7 th grade, we are developing 
instructional materials and professional development supports. To date these materials are in 
draft form and have been developed with input and advice from five expert middle- school 
math teachers as well as reviewed by a panel of mathematics experts recruited specifically 
for this task. As with the 6 th grade materials, knowledge from teaching experience, research 
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on teaching in these areas, and information gathered during the pilot testing year all play a 
role in developing these instructional materials. 

Supplementary Research Activities 

Following is a brief update of several supplementary strands of work undertaken as part 
of CRESST activities during the 2007-08 school year. This work includes validation of 
teacher math knowledge measures, continued investigation of how computer-based 
assessment strategies can support mathematics learning, investigation of states’ ELL 
assessment and accommodation practices (as part of CRESST supplementary assessment 
center grant), investigation of district contexts for assessment, and international applications 
of the POWERSOURCE work. Each of these strands is discussed separately. 

Validation Study of Teacher Math Knowledge Measures 

A validation study was conducted on several measures of teacher knowledge with 
respect to math. Validity was examined by comparing the performance of math (non- 
teaching) experts, expert math teachers, experienced math teachers, and novice math 
teachers. Group differences were explored for each task, and the relation among tasks was 
investigated. 

Participants. Eighty- six participants were recruited from several states through a 
teacher list-serve. Participants were recruited from several groups including two types of 
experts, math experts (n = 13) and teacher experts (n = 10), along with two groups of non- 
experts, novice teachers (n = 17) and experienced teachers (n = 46). The purpose of the 
different groups of participants was to ensure expertise in certain types of knowledge. 

Measures. All participants completed five tasks and a background survey. These tasks 
included the Study of Instructional Improvement (SII) instrument (Hill, Schilling, & Ball, 
2004), a ranking of student work (Baker, 2007), analysis of student work (Heritage, Kim, & 
Vendlinski, 2007), as well as a two-part knowledge mapping task (Delacruz et al., 2007) that 
was designed to measure content knowledge (KMP1) and math teaching knowledge (KMP2). 
The SII instrument, the student work ranking task, analysis of student work, and the first part 
of the knowledge mapping task were all designed to measure knowledge for teaching 
algebra. 

Summary of findings. Overall, we found evidence that math experts consistently 
scored highest on tasks that required high content knowledge and teacher experts scored 
highest on the student response analysis task, a measure of knowledge for teaching algebra. 
In addition, we found evidence that the knowledge mapping task was sensitive to differences 
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in both math content knowledge and math teaching knowledge, consistent with prior results 
(Delacruz et al., 2007). 

Implications. One important implication of this work for POWERS OURCE 0 was that 
this set of tasks can be used for evaluation of teachers participating in professional 
development. The SII, the knowledge mapping task, and the student response analysis task 
could be used as a set, or the knowledge mapping task could be used separately from the 
other tasks in order to efficiently investigate both content knowledge of teachers (KMP1) and 
knowledge of teaching algebra (KMP2). These findings supported the integration of the 
Knowledge Map task into the teacher outcome measures for the 2007-08 POWER S O URGE 
implementation study. 

A second important implication of this work is the potential application of these 
measures beyond the POWERSOURCE study by both researchers and practitioners looking 
for measures of teacher knowledge of math at the middle school level, with the suggestion 
that non-traditional approaches to teacher knowledge assessment, such as the knowledge 
mapper, can provide useful information about teacher learning. 

Assessment and Accommodation Practices for ELLs 

As described in our earlier progress report (Baker, 2007), CRESST received a 
supplemental center grant focused on state ELL assessment practices. This research project 
addresses the validity of assessments used to measure the performance of ELLs, such as 
those mandated by the NCLB Act of 2001 (2002). The goals of the research are to help 
educators understand and improve ELL performance by investigating the validity of their 
current assessments, and to provide states with much needed guidance to improve the validity 
of their English language proficiency (ELP) and academic achievement assessments for ELL 
students. The research has three phases. In the first phase of the study, the researchers 
analyzed existing data and documents in three states to understand the nature and validity of 
states’ current assessment practices and their priority needs. The analyses and results of 
phase 1 focused on the following six areas: (1) the language demands exhibited on state 
content-area and ELP assessments, (2) the identification of items that function differentially 
for ELL subgroups and the characteristic of those items, (3) achievement gaps among the 
subgroups of ELL students (e.g., ELLs, re-designated ELLs) compared to non-ELL students, 
(4) the relationship between the ELP and content-area assessment scores, (5) the factors 
related to the redesignation status, and (6) accommodation practices. 

The first phase of the project was exploratory in that the researchers identified key 
validity issues by examining the existing data and formulating research areas where further 
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investigation is needed. In the second phase of the research, currently underway, the 
researchers have been working with two states to develop and implement research studies to 
support the validity of ELL assessment practices within the states. Results from the first 
phase of the project and plans for the second phase were detailed in the phase 1 project report 
previously provided to IES. In the third phase of the research, the researchers will use what 
was learned in the first two phases of the project to develop specific guidelines on which 
states may base their ELL assessment policy and practice. 

Technology-Based Studies 

Several inter-related studies were undertaken during the 2007-08 project year that 
focused on the use of computer technology as part of the math assessment process. Some of 
these activities specifically included POWERSOURCE materials (i.e., instructional 
materials, Checks for Understanding), and others focused on the development and testing 
technology applications that could be integrated into subsequent iterations of 
POWERSOURCE 0 and POWERSOURCE 0, s web-based support materials. 

Another unifying aspect of these studies’ design was that they all incorporated a 
computer-based classroom response system technology. The software, called 31 (Interactive, 
Individualized Instruction), was originally developed in collaboration with the UCLA School 
of Engineering to provide the capability for students to respond (by typing) to instructor 
prompts (Encarnacao, Espinosa, Au, Chung, Johnson, & Kaiser, 2008). The general goal is to 
promote student engagement and problem solving with the content; to promote student 
interaction with the instructor; and lastly, to provide instructors with real-time student 
response data feedback about students’ understanding of the material. 

This approach incorporates key elements of an effective formative assessment system: 
(a) the capability of the system to provide good information about what students know and 
do not know; (b) the use of that information by instructors to provide feedback to students 
about their performance; and (c) the use of that information by instructors to adjust 
instruction. As with effective tutoring (Bloom, 1984) and instructional techniques that 
promote interaction and engagement (Hake, 1998), the fundamental enabling capability is the 
bidirectional flow of accurate information between student and instructor. Students solve 
problems and ask questions — while the instructor observes performance, interprets student 
actions, questions, and performance, and remediates as needed. 

Two research initiatives were undertaken as part of this technology strand of work 
during the current reporting period. The first initiative involved studies testing the response 
system in various middle school settings to examine which aspects of 31 — originally 
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developed for use in a higher education setting — would need to be adjusted for middle school 
mathematics use. Our middle school settings varied with respect to setting (summer school, 
regular school), scheduling (block schedule, normal schedule), teachers’ experience with 
technology (very inexperienced, typical level of experience), and experience in teaching the 
content (all experienced). The second initiative was to gather validity evidence for a student 
survey that could be used to measure students’ perceptions of classroom processes afforded 
by such a response system. 

Use of Interim Assessment Data/District Contexts 

This research activity takes a broader contextual approach to interim assessment use — 
examining the ways in which middle school mathematics teachers use the data provided by 
POWERSOURCE and other types of interim assessments, and how the features of the 
interim assessments are related to data use. The project is being conducted simultaneously at 
three sites — Central Colorado, Southern California, and Northern California (with lead 
researchers including Dr. Lorrie Shepard of the University of Colorado; Dr. Hilda Borko of 
Stanford University, and Dr. Brian Stecher of RAND). Districts were selected that had 
invested in teacher professional development around formative assessment or had installed 
formal interim assessment systems. Although three sites are involved, the project operates as 
a single study with a common design and procedures. 

During the past project year, the team has engaged in four major activities: instrument 
development, pilot testing, recruitment, and data collection. Three interview guides were 
developed for use in collecting data from district-level administrators, school principals and 
classroom teachers. The classroom teacher interview took place in two stages: initially by 
telephone and subsequently in person. Pilot testing was completed for these instruments with 
appropriate respondents in California and Colorado, and revisions were made based on the 
results of those pilots. After the pilot testing, the team recruited up to three middle schools in 
each district to participate in the study; within each school we recruited up to three 
mathematics teachers. The number of schools and teachers varied from site to site, based on 
local conditions. In January, principal and teacher interviews began; by the end of the school 
year, the team had completed approximately 20 interviews with school administrators and 
about 20 interviews with mathematics teachers. This summer the team will begin analyzing 
the interview transcripts (and the classroom artifacts we obtain during our second school 
interviews). In the fall will continue the interviews with the remaining school administrators 
and teachers. 
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International Applications of POWERSOURCE® 

As described in our previous progress report (Baker, 2007), through the extension of 
previous professional relationships, a collaboration with Korean educators who wish to test 
out POWERSOURCE in their country has been developed. In 2007, these Korean partners 
translated POWERSOURCE materials into Korean, including teacher handbooks, 
assessments, and instructional materials (Choe, 2007). Based on these materials, in late 2007 
teacher training was carried out by Korean researchers in Korea with 25 middle school 
teachers and 5 elementary school teachers. Over the spring of 2008, Korean researchers 
selected 21 teachers from those who attended the 2007 teacher training to implement 
POWERSOURCE in their grade classrooms; the same number of teachers was selected as a 
control group from the same school as the experimental group in this study. For the 
experimental group, four translated POWERSOURCE modules will be implemented over 
the summer 2008 (including Checks for Understanding and instructional materials); whereas, 
for the control group the teachers’ pre-existing formative assessment tools will be used. 

Plans for 2008-09 

Currently, we are beginning data analysis of data collected during the 2007-08 school 
year, including the student transfer measure and the multiple teacher outcomes described in 
this paper. Additionally, we will analyze the Checks for Understanding completed by the 
POWERSOURCE group teachers, both in terms of statistical quality of the items and to 
track student scores across the school year. We will also analyze state test data outcomes as 
they are made available by the districts including, when available, sub scale scores of state 
mathematics items. 

The focus for project implementation during the 2008-09 school year will be continuing 
the experimental (random assignment) study of POWERSOURCE impact begun in 2007- 
OS. Specifically, in addition to continuing the study at the 6 th grade level, we plan to add the 
7 th grade teachers in the participating districts to the study (note that, depending on district 
configuration, there may be some overlap in sample in cases where the same teachers teach 
both 6 th and 7 th grade math). The study will utilize a similar design and instrumentation to 
what was described above regarding the 2007-08 study, with student and teacher outcome 
instruments adapted to reflect 7 th grade content as applicable. We will also continue our 
recruiting efforts to add districts to the project and increase overall sample size. During the 
2008-09 school year we will also develop 8 th grade materials (including Checks for 
Understanding, instructional activities, and professional development resources), with the 
goal of expanding the study to 8 th grade in the 2009-10 school year. 
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Data collection and analysis for the supplemental ELL assessment study will continue. 
Another core planned set of supplemental activities is our leadership strand of work. Our 
leadership activities intend to support states and districts in their desire to develop coherent 
instructional programs to engage in standards-based reform; moreover, the work will focus in 
two areas. Lirst, it will focus on the collaborative development of methodology and annotated 
examples that practitioners and contractors can use to align instruction and assessment 
developmentally with key priorities for student capability in mathematics as well as with 
standards. The methodology seeks deeper understanding and communication of the learning 
demands, inherent standards, and the developmental progressions that are essential to 
accomplishing key standards. The methodology lays out a systematic framework describing 
these learning demands and progression, rather than simply working backward from one 
existing test. Products from the proposed effort will include software with embedded tutorials 
for conducting alignment analyses, paper and poster illustrations, and the results of 
workshops and webinars held with experts in math, math education, test developers, and 
other researchers as well as with the practitioner and policy communities. 

Secondly, to support the ongoing quality of current assessments, we propose a series of 
working conferences — connected to wider audiences through webinars — to clarify the 
specific criteria that states, local districts, and schools should address in selecting and 
refining these assessments and in order to compare and leam from promising systems for 
which there are positive data (e.g., POWERSOURCE , assessments of the Center for Data- 
Driven Reform in Education). We also propose making the criteria and examples of their 
application available interactively on the CRESST website; using web 2.0 interactive 
capability to promote attention and dialogue on the quality of existing assessment practices; 
and lastly responding to state, district, and local district queries on improving their systems. 
These efforts will concentrate on areas of recent CRESST research — formative assessment as 
well as ELP assessment — and provide a forum for sharing the best of current knowledge 
from CRESST and other leading efforts across the country. 
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Appendix A: 

Additional Item Analysis Results 




1. RNE: 05-06 Pilot Study 



FORM 


ITEM 


Short/Extended 


etc 


reliability 


Alpha 

if Item Deleted 


RNE-5 


rnebt4 


short item 






0.647 


(N=55) 


rnebt6 


short item 






0.661 




rnpw2a 


short item 






0.696 




rnpw2b 


short item 




0.687 


0.611 




rnpw2c 


short item 






0.612 




mepwel_a 


extended item 






0.664 




mepxveljb 


extended item 






0.670 


RNE-7-5 


rnebtl 


short item 






0.518 


(N=142) 


rnebt6 


short item 






0.504 




rneex6_a 


extended item 


meex6_a( i ) 


0.535 


0.480 




rneex6_b 


extended item 


meex6_a( ii ) 


0.466 




rneex6_c 


extended item 


rneex6_b( i ) 




0.508 




rneex6_d 


extended item 


rneex6_b( ii ) 




0.460 


RNE-8 


rnebt2 


short item 






0.453 


(N=77) 


rnebt5 


short item 




0.597 


0.567 




mewp2_a 


extended item 




0.582 




mewp2_b 


extended item 






0.493 


RNE-A4 


rnebtl 


short item 






0.635 


(N=127) 


rnebt2 


short item 




0.674 


0.622 




meexl_a 


extended item 




0.560 




meexljb 


extended item 






0.612 


RNE-B4 


rnebt2 


short item 


not "rnebt3" 




0.635 


(N=129) 


rnebt4 


short item 




0.707 


0.686 




meex2_a 


extended item 




0.679 




meex2_b 


extended item 






0.565 


RNE-C3 


rnebt5 


short item 






0.669 


(N=124) 


rnebt6 


short item 






0.717 




meex3_a 


extended item 




0.718 


0.665 




meex3_b 


extended item 






0.627 




meex3_c 


extended item 






0.669 



1 





2. RNE: 06-07 Pilot Study 



FORM ITEM Short/Extended 

RNE- 13 RN-BT-10 short item 

(N=107) RN-BT-4 short item 

RN-BT-8 short item 

RNE-14 RN-BT-1 short item 

(N=98) RN-BT-8 short item 

RNE-15 RN-BT-5 short item 

(N=101) RN-BT-8 short item 

RN-PW-2a short item 

RN-PW-2b short item 

RN-PW-2c short item 

RN-BT-7 short item 

RNE-16 RN-BT-6 short item 

(N=74) RN-BT-8 short item 

RN-PW-3a short item 

RN-PW-3b short item 

RN-PW-3c short item 

RNE-17 RN-BT-1 short item 

(N=88) RN-BT-2 short item 

RN-PW-4a short item 

RN-PW-4b short item 

RN-PW-4c short item 





3. PA: 05-06 Pilot Study 



FORM 


ITEM 


Short/Extended 


reliability 


Alpha 

if Item Deleted 


PA-A v3 
(PA-1 v3) 
(N=58) 


PA-BT-3 


short item 


0.443 


0.260 


PA-BT-4 


short item 


0.320 


PA-PW-3a 


short item 


0.268 


PA-PW-3b 


short item 


0.224 


PA-PWE-1 


extended item 


0.682 


PA-2 v4 
& 

PA-B v4 
(N=l 14) 


PA-BT-3 


short item 


0.689 


0.675 


PA-BT-4 


short item 


0.693 


PA-PW-2a 


short item 


0.576 


PA-PW-2b 


short item 


0.550 


PA-PW-2c 


short item 


0.568 


PA-PWE-2 


extended item 


0.765 


PA-C v2 
(PA-3 v2) 
(N=52) 


PA-BT-3 


short item 


0.558 


0.540 


PA-BT-4 


short item 


0.448 


PA-EX-la 


extended item 


0.521 


PA-EX-lb 


extended item 


0.426 


PA-F v3 
(PA-6 v6) 
(N=25) 


PA-BT-5 


short item 


0.558 


0.537 


PA-BT-6 


short item 


0.692 


PA-PW-4a 


short item 


0.287 


PA-PW-4b 


short item 


0.264 


PA-PW-4c 


short item 


0.255 


PA-PWE-1 


extended item 


0.719 


PA-G v3 
(PA-7 v3) 
(N=55) 


PA-BT-3 


short item 


0.551 


0.486 


PA-BT-4 


short item 


0.510 


PA-EX-2a 


extended item 


0.408 


PA-EX-2b 


extended item 


0.506 


PA-H v3 
(PA-8 v3) 
(N=57 ) 


PA-BT-3 


short item 


0.371 


0.398 


PA-BT-7 


short item 


0.136 


PA-PW-5a 


short item 


0.332 


PA-PW-5b 


short item 


0.370 


PA-PWE-3 


extended item 


0.331 


PA-9 v3 
(N=54) 


PA-BT-8 


short item 


0.682 


0.642 


PA-BT-9 


short item 


0.734 


PA-PW-4a 


short item 


0.538 


PA-PW-4b 


short item 


0.538 


PA-PW-4c 


short item 


0.546 


PA-PWE-3 


extended item 


0.760 


PA-10 v3 


PA-BT-10 


short item 


0.725 


0.667 
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4. SE: 05-06 Pilot Study 



FORM 


ITEM 


Short/Extended 


reliability 


Alpha 
if Item 
Deleted 


SE-1 v3 
(SE-A3) 
(N=87 ) 


SE-BT-1 


short item 


0.470 


0.531 


SE-BT-2 


short item 


0.429 


SE-PW-l-a 


short item 


0.340 


SE-PW-l-b 


short item 


0.361 


SE-PWE-1 


extended item 


0.387 


SE-B v4 
(SE-2-4) 
(SE-B7) 
(N=80) 


SE-BT-1 


short item 


0.561 


0.604 


SE-BT-2 


short item 


0.473 


SE-PW-2-a 


short item 


0.424 


SE-PW-2-b 


short item 


0.603 


SE-PWE-2 


extended item 


0.371 


SE-3 vl 
(N=17) 


SE-BT-1 


short item 


-0.226 


-0.402 


SE-BT-2 


short item 


0.118 


SE-EX-1 


extended item 


-0.201 


SE-D v4 
(SE-4-4) 
(N=59) 


SE-BT-3 


short item 


0.545 


0.549 


SE-BT-4 


short item 


0.518 


SE-PW-l-a 


short item 


0.523 


SE-PW-l-b 


short item 


0.415 


SE-PWE-1 


extended item 


0.426 


SE-5 v4 
(N=78) 


SE-BT-5 


short item 


0.512 


0.445 


SE-BT-6 


short item 


0.556 


SE-PW-l-a 


short item 


0.388 


SE-PW-l-b 


short item 


0.407 


SE-PWE-1 


extended item 


0.467 


SE-7 v4 
(N=121) 


SE-BT-1 


short item 


0.712 


0.756 


SE-BT-5 


short item 


0.711 


SE-WP-2-a 


extended item 


0.623 


SE-WP-2-b 


extended item 


0.622 


SE-WP-2-c 


extended item 


0.663 


SE-WP-2-d 


extended item 


0.668 


SE-WP-2-e 


extended item 


0.687 


SE-8 vl 
(N=109 ) 


SE-BT-5 


short item 


0.687 


0.654 


SE-BT-6 


short item 


0.737 


SE-PW-3-a 


short item 


0.616 


SE-PW-3-b 


short item 


0.617 


SE-PW-3-c 


short item 


0.589 


SE-EX-3 


extended item 


0.643 
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5. SE: 06-07 Pilot Study (extended items are not scored) 



FORM 


ITEM 


Short/Extended 


reliability 


Alpha 

if Item Deleted 


SE-15 

(N=73) 


SE-BT-7 


short item 


-0.266 




SE-WP-7 


extended item 






SE-EX-10-a 


short item 






SE-EX-10-b 


extended item 






SE-EX-10-c 


extended item 






SE-16 

(N=48) 


SE-BT-8 


short item 


-0.216 




SE-WP-8 


extended item 






SE-EX-1 1-a 


short item 






SE-EX-1 1-b 


extended item 






SE-EX-1 1-c 


extended item 






SE-17 

(N=35) 


SE-BT-9 


short item 


0.442 


0.140 


SE-WP-9-a 


short item 




0.404 


SE-WP-9-b 


extended item 






SE-WP-9-c 


extended item 






SE-EX-1 2-a 


short item 




0.459 


SE-EX-12-b 


extended item 






SE-EX-12-c 


extended item 






SE-18 

(N=18) 


SE-BT-7 


short item 


0.064 


0.264 


SE-WP-9-a 


short item 




-0.052 


SE-WP-9-b 


extended item 






SE-WP-9-c 


extended item 






SE-EX-1 3-a 


short item 




-0.127 


SE-EX-13-b 


extended item 






SE-EX-13-c 


extended item 






SE-19 

(N=39) 


SE-BT-8 


short item 


0.592 




SE-WP-7 


extended item 






SE-EX-14-a 


short item 






SE-EX-14-b 


extended item 






SE-EX-14-c 


extended item 






SE-20 

(N=80) 


SE-BT-9 


short item 


0.045 




SE-WP-8 


extended item 






SE-EX-15-a 


short item 






SE-EX-15-b 


extended item 






SE-EX-15-c 


extended item 









SE-BT-7 


short item 


SE-WP-8 


extended item 


SE-EX-16-a 


short item 


SE-EX-16-b 


extended item 


SE-EX-16-c 


extended item 


SE-BT-8 


short item 


SE-WP-9-a 


short item 


SE-WP-9-b 


extended item 


SE-WP-9-c 


extended item 


SE-EX-10-a 


short item 


SE-EX-lO-b 


extended item 


SE-EX-lO-c 


extended item 


SE-BT-8 


short item 


SE-WP-7 


extended item 


SE-EX-1 1-a 


short item 


SE-EX-1 1-b 


extended item 


SE-EX-1 1-c 


extended item 




IRT Tables 



1. RNE 05-06 & 06-07 Pilot 



item 


b 


SE(b) 


taul 


tau2 


tau3 


tau4 


G 2 


df 


p-value 


INFIT 

(MNSQ) 


OUTFIT 

(MNSQ) 


RNBTl 


-1.52 


0.09 










11.72 


5 


0.04 


1.07 


3.46 


RNBT2 


-1.11 


0.08 










35.91 


6 


0.00 


1.00 


1.05 


RNBT4 


-0.65 


0.08 










10.38 


6 


0.11 


1.17 


1.75 


RNBT5 


-1.01 


0.09 










28.74 


5 


0.00 


0.96 


1.07 


RNBT6 


-0.72 


0.07 










38.96 


7 


0.00 


1.04 


1.34 


RNBT7 


-1.48 


0.19 










0.00 


0 


0.00 


0.97 


0.74 


RNBT8 


-0.25 


0.08 










25.33 


8 


0.00 


1.11 


1.10 


RNBT10 


-3.10 


0.54 










0.00 


0 


0.00 


0.99 


1.13 


RNPW2A 


0.70 


0.12 










10.36 


6 


0.11 


1.00 


1.36 


RNPW2B 


0.78 


0.12 










27.31 


7 


0.00 


0.79 


0.62 


RNPW2C 


0.81 


0.13 










26.49 


7 


0.00 


0.86 


0.71 


RNPW3A 


-0.41 


0.16 










1.73 


2 


0.42 


0.87 


0.96 


RNPW3B 


-1.06 


0.19 










0.00 


0 


0.00 


0.96 


0.84 


RNPW3C 


-0.50 


0.16 










4.07 


2 


0.13 


0.83 


0.80 


RNPW4A 


0.02 


0.15 










9.05 


4 


0.06 


0.82 


0.62 


RNPW4B 


-1.07 


0.16 










0.20 


2 


0.90 


1.06 


0.86 


RNPW4C 


-0.48 


0.15 










6.80 


3 


0.08 


0.74 


0.67 


RNEEX1_A 


-0.61 


0.07 


-0.53 


-0.01 


0.55 




10.06 


3 


0.02 


0.61 


0.82 


RNEEX1_B 


1.06 


0.08 


1.39 


0.34 


-1.73 




92.64 


11 


0.00 


0.82 


0.67 


RNEEX2_A 


0.24 


0.08 


-0.17 


0.17 






18.87 


7 


0.01 


1.18 


1.19 


RNEEX2_B 


0.05 


0.05 


-0.37 


0.11 


0.06 


0.20 


8.72 


6 


0.19 


0.49 


0.53 


RNEEX3_A 


0.50 


0.07 


-0.05 


0.90 


-0.85 




16.85 


12 


0.16 


1.04 


1.00 


RNEEX3_B 


0.06 


0.08 


-0.26 


0.26 






21.46 


6 


0.00 


0.76 


0.80 


RNEEX3_C 


1.42 


0.09 


0.63 


0.50 


-1.14 




19.27 


9 


0.02 


0.96 


0.71 


RNEEX6_A 


0.74 


0.12 










20.13 


4 


0.00 


0.91 


2.81 


RNEEX6_B 


1.29 


0.11 


-0.47 


0.47 






18.68 


6 


0.01 


0.86 


6.02 


RNEEX6_C 


0.94 


0.13 










9.27 


5 


0.10 


1.25 


2.12 


RNEEX6_D 


1.61 


0.15 


-0.74 


0.74 






11.00 


6 


0.09 


0.89 


0.15 


RNEPWE1_ 


0.94 


0.16 


0.44 


-0.44 






3.45 


4 


0.49 


1.03 


1.02 


RNEPWE_1 


1.02 


0.15 


0.23 


-0.77 


0.55 




3.54 


4 


0.47 


1.11 


0.86 


RNEWP2_A 


0.71 


0.17 










10.77 


4 


0.03 


1.17 


4.61 


RNEWP2__B 


1.48 


0.13 


0.02 


0.70 


-0.71 




8.93 


5 


0.11 


0.90 


0.23 




2. RNE 06-07 Field 



item 


b 


SE(b) 


taul 


tau2 


tau3 


G 2 


df 


p-value 


INFIT 

(MNSQ) 


OUTFIT 

(MNSQ) 


RNBT7 


-2.48 


0.08 








8.50 


5 


0.13 


1.19 


2.15 


RNBT8 


-0.57 


0.02 








258.57 


10 


0.00 


0.88 


0.79 


RNBT9 


-0.50 


0.02 








275.73 


10 


0.00 


0.87 


0.78 


RNBT4 


-0.83 


0.03 








85.63 


9 


0.00 


0.99 


0.94 


RNBT2 


-0.83 


0.03 








157.39 


9 


0.00 


0.94 


0.83 


EX8 


0.76 


0.02 


0.24 


-1.03 


0.79 


113.72 


25 


0.00 


1.05 


1.22 


EX9 


-0.22 


0.01 


-1.26 


0.45 


0.81 


96.68 


23 


0.00 


1.24 


2.10 


EX1A 


-0.11 


0.01 


-0.66 


-0.02 


0.68 


117.15 


26 


0.00 


0.94 


0.98 


EX1B 


0.97 


0.02 


0.60 


-0.19 


-0.41 


330.23 


27 


0.00 


0.89 


0.79 


PWE_2 


0.81 


0.02 


-0.20 


0.44 


-0.24 


32.37 


25 


0.15 


1.03 


1.22 


EX6al 


0.15 


0.04 








44.15 


10 


0.00 


1.12 


1.28 


EX6a2 


1.08 


0.03 


-0.50 


0.02 


0.48 


25.07 


20 


0.20 


1.08 


1.50 


EX6bl 


0.70 


0.04 








66.66 


10 


0.00 


1.12 


1.75 


EX6b2 


1.26 


0.04 


-0.76 


0.37 


0.39 


31.88 


16 


0.01 


0.93 


1.23 
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3. PA 05-06 Pilot 



item 


b 


SE(b) 


taul 


tau2 


tau3 


G 2 


df 


p-value 


INFIT 

(MNSQ) 


OUTFIT 

(MNSQ) 


pabt3 


-0.85 


0.07 








9.67 


4 


0.05 


0.99 


1.41 


pabt4 


0.10 


0.08 








10.93 


3 


0.01 


1.19 


2.35 


pabt5 


-0.63 


0.29 








0.00 


0 


0.00 


0.96 


0.82 


pabt6 


0.29 


0.28 








0.40 


2 


0.82 


1.56 


1.59 


pabt7 


-0.37 


0.10 








14.01 


3 


0.00 


0.82 


0.77 


pabt8 


-0.34 


0.19 








1.28 


2 


0.53 


0.97 


0.74 


pabt9 


0.27 


0.19 








0.92 


2 


0.64 


1.24 


1.56 


pabtlO 


-0.65 


0.08 








23.11 


4 


0.00 


1.05 


1.16 


pabtl 1 


-0.27 


0.07 








33.46 


4 


0.00 


0.98 


1.1 


papw2a 


0.69 


0.14 








7.52 


3 


0.06 


0.68 


0.46 


papw2b 


0.66 


0.14 








13.91 


3 


0.00 


0.57 


0.38 


papw2c 


0.80 


0.15 








15.10 


3 


0.00 


0.6 


0.37 


papw3a 


-0.12 


0.18 








0.74 


1 


0.39 


0.8 


0.7 


papw3b 


-0.41 


0.19 








0.95 


1 


0.33 


0.84 


0.59 


papw4a 


-0.35 


0.16 








10.78 


2 


0.01 


0.7 


0.51 


papw4b 


-0.01 


0.16 








9.95 


2 


0.01 


0.74 


0.61 


papw5a 


0.29 


0.18 








0.41 


2 


0.82 


1.25 


1.42 


papw6a 


0.91 


0.28 








0.02 


2 


0.98 


1.27 


1.54 


papw6b 


0.56 


0.25 








4.45 


2 


0.11 


0.74 


0.59 


papw6c 


0.67 


0.26 








0.37 


2 


0.83 


1.08 


1.11 


papw7a 


-0.06 


0.16 








17.09 


4 


0.00 


0.84 


0.73 


papw7b 


0.15 


0.16 








18.32 


4 


0.00 


0.83 


0.66 


papw7c 


-0.29 


0.16 








19.90 


4 


0.00 


0.76 


0.7 


papw8a 


-0.04 


0.26 








1.03 


2 


0.60 


0.82 


0.79 


papw8b 


0.46 


0.28 








0.03 


2 


0.98 


1.13 


1.26 


papw8c 


0.91 


0.32 








1.01 


2 


0.61 


0.77 


0.49 


paex l_a 


1.59 


0.14 


0.59 


0.19 


-0.79 


1.85 


4 


0.77 


0.82 


0.73 


paexl_b 


0.50 


0.09 


-0.69 


-0.73 


1.42 


0.83 


2 


0.66 


0.47 


0.96 


paex2_a 


1.20 


0.10 


0.13 


-0.13 




15.62 


5 


0.01 


1.2 


1.12 


paex2_b 


0.59 


0.06 


-0.13 


-1.08 


1.21 


5.04 


4 


0.28 


0.68 


0.94 


paex3_a 


-1.60 


0.18 








2.40 


1 


0.12 


1.36 


1.58 


paex3_b 


1.18 


0.15 








2.85 


3 


0.42 


1.05 


0.88 


paex3_c 


1.32 


0.10 


1.34 


-0.35 


-0.99 


12.34 


7 


0.09 


1.08 


0.99 


paex6 


1.15 


0.11 


0.62 


-1.13 


0.51 


4.43 


4 


0.35 


1.08 


1.46 


pap we 1 


1.67 


0.18 


-0.82 


0.82 




23.77 


3 


0.00 


1.22 


9.9 


papwe2 


2.06 


0.19 


-0.21 


0.21 




15.42 


4 


0.00 


1.41 


5.89 


papwe3 


1.81 


0.13 


-0.25 


0.96 


-0.71 


6.09 


4 


0.19 


0.79 


3.09 




4. PA 06-07 Field 



item 


b 


SE(b) 


taul 


tau2 


tau3 


G 2 


df 


p-value 


INFIT 

(MNSQ) 


OUTFIT 

(MNSQ) 


PABTIJR 


-0.25 


0.02 








51.04 


10 


0 


1.09 


1.22 


PABT13_1 


-0.05 


0.02 








81.43 


10 


0 


0.96 


0.99 


PABT9JR 


-0.39 


0.02 








33.23 


9 


0 


1.08 


1.1 


PAPW8A_1 


0.32 


0.02 








108.31 


10 


0 


1.07 


1.31 


PAPW8B„1 


0.57 


0.02 








195.87 


10 


0 


0.85 


1.02 


PAPW8C_1 


0.56 


0.02 








163.49 


10 


0 


0.89 


1.25 


PABT2_2R 


-0.98 


0.02 








46.02 


8 


0 


0.93 


0.94 


PABT3_2R 


-1.15 


0.02 








72.56 


8 


0 


0.86 


0.75 


PABT12_2 


-1.00 


0.02 








49.50 


8 


0 


0.93 


0.78 


PAPW4A_2 


-1.21 


0.02 








89.94 


7 


0 


0.85 


0.58 


PAPW4B_2 


-0.67 


0.02 








233.04 


9 


0 


0.71 


0.58 


PAPW4C_2 


-0.58 


0.02 








184.22 


9 


0 


0.76 


0.75 


PAEXla 


1.61 


0.03 


-0.71 


0.71 




85.05 


13 


0 


1.22 


5.3 


PAEXlb 


0.34 


0.02 


-0.36 


0.36 




25.33 


13 


0.02 


1.14 


1.6 


PAPWE3 


1.84 


0.02 


0.67 


0.80 


-1.47 


148.80 


20 


0 


1.46 


2.64 


PAEX6a 


1.59 


0.02 


0.77 


0.65 


-1.42 


39.04 


21 


0.01 


1.12 


1.45 


PAEX6b 


-0.37 


0.03 








137.70 


8 


0 


1.58 


2.01 



11 




5. SE 05-06 & 06-07 Pilot 



item 


b 


SE(b) 


taul 


tau2 


tau3 


G 2 


df 


p-value 


INFIT 

(MNSQ) 


OUTFIT 

(MNSQ) 


sebtl 


-2.12 


0.18 








0.02 


1 


0.87 


1.31 


1.49 


sebt2 


-0.31 


0.10 








0.38 


2 


0.83 


1.05 


0.98 


sebt3 


-1.80 


0.33 








0.00 


0 


0.00 


1.10 


3.25 


sebt4 


-2.01 


0.37 








0.00 


0 


0.00 


0.92 


0.71 


sebt5 


0.13 


0.08 








2.74 


4 


0.61 


1.25 


1.63 


sebt6 


-1.75 


0.17 








1.55 


1 


0.21 


1.27 


1.80 


sebt7 


-1.15 


0.15 








10.49 


2 


0.01 


1.05 


2.23 


sebt8 


-1.38 


0.15 








14.37 


2 


0.00 


0.78 


0.56 


sebt9 


-0.33 


0.15 








4.08 


2 


0.13 


0.93 


0.88 


sepwla 


-0.81 


0.10 








19.29 


2 


0.00 


0.80 


0.80 


sepwlb 


-0.77 


0.10 








20.70 


2 


0.00 


0.73 


0.64 


sepw2a 


-0.01 


0.15 








3.44 


2 


0.18 


0.95 


1.13 


sepw2b 


-0.72 


0.17 








2.01 


2 


0.37 


1.25 


1.26 


sepw3a 


-0.29 


0.13 








1.26 


2 


0.54 


0.90 


0.78 


sepw3b 


0.44 


0.14 








2.70 


3 


0.44 


1.04 


0.92 


sepw3c 


-0.44 


0.13 








4.27 


2 


0.12 


0.76 


0.61 


seexl 


0.49 


0.25 


1.67 


-0.63 


-1.04 


0.59 


1 


0.45 


0.85 


0.80 


seex3 


0.36 


0.07 


0.07 


0.52 


-0.59 


2.57 


5 


0.77 


1.12 


0.93 


seexlOa 


-1.09 


0.17 








9.78 


2 


0.01 


1.20 


0.87 


seexl la 


-1.62 


0.20 








0.00 


0 


0.00 


0.94 


0.61 


seexl 2a 


-2.03 


0.45 








0.00 


0 


0.00 


0.94 


0.49 


seexl 3a 


-0.49 


0.33 








0.01 


1 


0.87 


1.34 


1.79 


seexl 4a 


-1.09 


0.27 








0.00 


0 


0.00 


0.99 


0.98 


seexl 5a 


-1.56 


0.22 








0.00 


1 


0.91 


1.13 


1.02 


seexl 6a 


-1.85 


0.38 








0.00 


0 


0.00 


1.00 


1.00 


sepwel 


0.69 


0.10 








20.05 


3 


0.00 


1.06 


1.21 


sepwe2 


0.78 


0.12 


0.09 


-0.09 




6.51 


4 


0.16 


0.79 


0.70 


sewp2a 


-0.93 


0.13 


0.93 


-0.93 




2.50 


2 


0.29 


0.55 


0.45 


sewp2b 


-0.91 


0.13 


0.95 


-0.95 




2.92 


2 


0.23 


0.56 


0.46 


sewp2c 


1.38 


0.20 








2.75 


3 


0.43 


0.94 


0.74 


sewp2d 


1.53 


0.22 








1.41 


3 


0.71 


1.34 


9.90 


sewp2e 


0.07 


0.11 


0.65 


-0.65 




2.36 


5 


0.80 


1.21 


1.12 


sewp9a 


1.25 


0.20 








23.12 


3 


0.00 


1.01 


0.70 




6. SE 06-07 Field 



item 


b 


SE(b) 


taul 


tau2 


tau3 


G 2 


df 


P" 

value 


INFIT 

(MNSQ) 


OUTFIT 

(MNSQ) 


sebt2 


-0.21 


0.03 








246.52 


10 


0.00 


0.94 


0.92 


sebt3 


-3.35 


0.14 








0 


0 


0.00 


1.11 


2.44 


sebt7 


-1.35 


0.03 








28.5557 


9 


0.00 


1.14 


1.91 


sebt8 


-1.56 


0.04 








51.6484 


8 


0.00 


1.08 


1.90 


sepw2a 


0.01 


0.02 








162.165 


10 


0.00 


1.12 


1.31 


sepw2b 


-0.67 


0.03 








161.148 


10 


0.00 


0.99 


1.03 


sepw3a 


-0.38 


0.03 








352.773 


10 


0.00 


0.85 


0.83 


sepw3b 


0.33 


0.03 








262.473 


10 


0.00 


0.99 


1.11 


sepw3c 


-0.46 


0.03 








382.581 


10 


0.00 


0.83 


0.87 


seex6a 


-0.12 


0.03 


0.90 


-0.90 




206.559 


17 


0.00 


0.83 


0.82 


seexbb 


1.52 


0.03 


0.44 


0.10 


-0.55 


156.45 


20 


0.00 


0.80 


0.60 


seex9 


1.14 


0.03 


0.37 


-0.23 


-0.14 


53.9213 


24 


0.00 


1.07 


1.36 


sepwe2 


1.58 


0.04 


0.46 


-0.12 


-0.34 


45.2868 


19 


0.00 


1.08 


1.02 
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7. RA 06-07 Field 



item 


b 


SE(b) 


taul 


tau2 


tau3 


G 2 


df 


p-value 


INFIT 

(MNSQ) 


OUTFIT 

(MNSQ) 


rasebt9 


-1.03 


0.05 








41.28 


7 


0.00 


1.30 


1.98 


rasebtlO 


-1.65 


0.06 








6.00 


5 


0.31 


1.00 


1.53 


rasebtl 1 


-0.41 


0.04 








15.98 


8 


0.04 


0.97 


1.03 


rpabtl4a 


-0.66 


0.04 








46.30 


8 


0.00 


0.84 


0.69 


rpabtl4b 


-0.64 


0.04 








20.95 


8 


0.01 


1.02 


0.95 


rpabtl4c 


-0.24 


0.04 








78.73 


9 


0.00 


0.78 


0.68 


rpabtl5a 


-0.58 


0.04 








25.15 


8 


0.00 


0.90 


0.75 


rpabtl5b 


-0.42 


0.04 








26.25 


8 


0.00 


1.10 


1.15 


rarnbtl 1 


-1.62 


0.06 








26.07 


5 


0.00 


0.86 


0.87 


rarnbtl3 


-1.40 


0.05 








30.73 


7 


0.00 


0.83 


0.72 


rarnwp3 


-0.41 


0.02 


-0.55 


0.49 


0.06 


60.81 


17 


0.00 


1.25 


1.61 


rarnwp4 


0.75 


0.05 








44.99 


10 


0.00 


1.04 


0.99 


rarngpl 


-0.80 


0.05 








5.68 


8 


0.68 


1.01 


1.16 


rarnbtl 4 


0.01 


0.04 








22.24 


9 


0.01 


1.01 


0.96 


rapabtl6 


0.14 


0.07 








43.56 


8 


0.00 


0.69 


0.6 


rasewplO 


0.63 


0.04 


0.61 


-0.60 


-0.01 


16.08 


15 


0.38 


1.10 


1.31 


rasegpl 


-0.08 


0.06 


1.19 


-1.19 




21.43 


14 


0.09 


1.23 


1.27 


rapabtl8 


0.55 


0.05 


0.21 


-0.21 




18.76 


14 


0.17 


0.95 


0.93 


rapabtl7 


0.28 


0.07 








49.03 


9 


0.00 


0.78 


0.79 




8. FM 06-07 Field 



item 


b 


SE(b) 


taul 


tau2 


G 2 


df 


p-value 


INFIT 

(MNSQ) 


OUTFIT 

(MNSQ) 


mbt7 


-2.82 


0.11 






3.54 


2 


0.17 


1.17 


1.74 


mbt8 


-0.52 


0.04 






73.94 


4 


0.00 


1.36 


1.66 


rnpw3a 


0.09 


0.04 






41.04 


4 


0.00 


0.91 


0.79 


rnpw3b 


-1.01 


0.05 






7.60 


4 


0.11 


1.07 


5.8 


rnpw3c 


0.07 


0.04 






43.42 


4 


0.00 


0.87 


0.72 


mbtlO 


-0.54 


0.04 






35.59 


4 


0.00 


0.81 


0.74 


mbt2 


-0.70 


0.05 






15.54 


4 


0.00 


0.87 


0.78 


rnpw4a 


0.01 


0.04 






21.25 


4 


0.00 


0.97 


1.06 


rnpw4b 


-0.93 


0.05 






24.95 


4 


0.00 


1.13 


1.05 


rnpw4c 


-0.14 


0.04 






56.72 


4 


0.00 


0.8 


0.66 


rnpwe3 


2.60 


0.09 


0.69 


-0.69 


54.29 


8 


0.00 


1.15 


2.46 



15 




PABT1_1R PABT13_1 PABT9_1R PAPW8A_1 PAPW8B_1 
PAPW8C_1 PABT2_2R PABT3_2R PABT12_2 PAPW4A_2 
PAPW4B_2 PAPW4C_2 PAEXla PAEXlb PAPWE3 



PAEX6a 



PAEX6b 






4. Items in RA 06-07 Field Tests 




rasebt9 rasebtlO rasebtll rpabtl4a rpabtl4b 

rpabtl4c rpabtl5a rpabtl5b rarnbtll rarnbtl3 

rarnwp3 rarnwp4 rarngpl rarnbtl4 rapabtl6 

rasewplO rasegpl rapabt!8 rapabt!7 
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Introduction 



This handbook is divided into several parts. These parts reflect 
what you will be doing in each of the units. This handbook 
accompanies the second Powersource topic - properties of 
arithmetic. You will notice that all assessments begin with the 
number 2. So the first assessment, given prior to instruction 
is Checks for Understanding 2, the second one is Checks for 
Understanding 2a and the third is Checks for Understanding 2b. 

1 . Day 1 : Checks for Understanding 2 

The first Checks for Understanding for properties of arithmetic 
has ten questions covering the distributive property. You will 
administer this assessment to your students on Day 1. It should 
take your students about 20-25 minutes to complete this Checks 
for Understanding. 

2. Understanding the Results 2 

Once your students have completed Checks for Understanding 

2, you can tally their scores and enter them into the table on 
page 13. Information in this table will allow you to easily 
see which problems were answered incorrectly by the most 
students. You can look at the total incorrect for each item, and 
use the information about the items on pages 15 through 16 to 
help you plan your teaching of the content in Lesson 1. 

3. Day 2: Teach Lesson 1 

The instructional text on pages 18 and 33 is your guide to the 
content for each of two lessons on the distributive property. 
These lessons are designed as an example of how a teacher 
might teach a lesson on the distributive property. Please do not 
feel as if you have to adhere exactly to this example lesson. You 
do not have to say or do exactly what is laid out in the lesson. 
Rather this is an example of one way that you could teach this 
content. This is only a guideline and if you have your own ideas 
about how to teach this content, please feel free to use them. 
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Introduction 



The important thing is to make sure you are covering the concepts 
presented in the sample lesson. We hope that some of the examples 
and overheads will be useful as you help students to understand these 
concepts, and we recommend the use of these examples somewhere 
within your instruction. 

The lesson consists of the following: 

Material to say to the class: 

Have you heard of the distributive property? Can anyone 
sa Y say how it works? 



Material to write on the board: 



write 



2(3+4) = 2 • 3 + 2 • 4 



Overheads to show to the class: 

show Overhead 1 



4. Checks for Understanding 2a 

Once you have gone over the content in Lesson 1, you will administer 
Checks for Understanding 2a. This assessment contains items based 
on content taught in Lesson 1. The items focus on the distributive 
property. 

5. Understanding the Results 2a 

Using the chart on page 30, you can again record how many students in 
your class answered incorrectly on each item. If you have students who 
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Introduction 



did not understand some of the concepts in Lesson 1, you may choose 
to go back over some of the ideas presented in that lesson. The items on 
Check for Understanding 2a are comparable to items 1,3,4 and 8 on 
the Checks for Understanding 2. 

6. Day 3: Teach Lesson 2 

Lesson 2 is designed to be used either by individual students, as 
a group exercise, or as a lesson taught by you. It is up to you to 
decide how to use Lesson 2. You might have some students who 
need extra time on the concepts presented in Lesson 1. 

7. Checks for Understanding 2b 

Once you have gone over the content in Lesson 2, you will 
administer Checks for Understanding 2b to your students. 

This assessment is based on material covered in Lesson 2 on 
the distributive property. Students solve problems, and write 
explanations about the distributive property in Checks for 
Understanding 2b. 

8. Understanding the Results 2b 

Using the chart on page 44 you can record how many of your 
students answered incorrectly to a particular problem. If you have 
students who did not understand some of the concepts in Lesson 2, 
you may choose to go back over some of the ideas presented in that 
lesson. The items on Checks for Understanding 2b are comparable 
to items 1, 2, 5, 6 and 7 on Checks for Understanding 2. 

* See the diagram on the next page for an overview of the study. 
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!. Checks for Understanding on Lesson 2 




Administering 
Checks for 
Understanding 2 




Administering Checks for Understanding 2 



Please distribute the Check Your Understanding booklets 
to the students in your class in the order you found them 
in the envelope. Place any unused booklets back in the 
original envelope. 

Today we are going to do some short problems that will 
take about 20-25 minutes for you to complete. Some 
problems may ask you to figure out what the missing 
numbers are, and some may ask you to explain your thinking. 

say Read each problem and write the answer that you think 
is best. If there are any words that you are confused 
about, circle them and put a question mark next to the circle. 

^ These problems are intended to last no more than 20-25 
minutes. Please stop your students after 20-25 minutes, 
even if they are not finished. 

^ When the students are working on the problems, if they 
ask a question, please make a note of it on the pages 
labeled Teacher Notes of Student Questions. Please do 
not provide information to a student that would help him 
or her answer a problem. 

^ In the CRESST folder, you will find yellow copies of the 
Check Your Understanding forms your students are taking. 
If you have suggestions for changes, please write them 
directly on these forms. 

After students have worked on the problems for 20-25 
minutes, please put all of the students’ booklets back in 
their original envelope. 
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Understanding 
the Results-Checks 
for Understanding 2 




Understanding the Results 2 



Checks for Understanding 2 is based on material you will be 
covering in Lessons 1 and 2. These lessons focus on the distrib- 
utive property. Below is the answer key to Checks for 
Understanding 2. 



Check Your 
Understanding 



Answer all questions below. Be sure to show all your work. 



O 7 (4 + 2) = 7 



•4+7*2 



© 4 (y + 2) 

For this problem write only the first step that you would write if you were simplifying 
this problem. 



possible answers: 4y 

(4 • y) + (4 • 2) 
4 • / + 8 
4y + 8 



Go to next page. 
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Understanding the Results 2 



Check Your 
Understanding 



© Show how you can use the distributive property to rewrite this equation: 
3(2 + 4 ) = 



3 • 2 + 3 • 4 



© 6(3 + 1 ) = 6-3 |0 



6 • 1 



Go to next page. 
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Understanding the Results 2 



Check Your 
Understanding 



Complete the number sentence below. 



© 



3 (x + 5) = ( 3 -jc) + (3*5) 



Using what you know about mathematical principles, explain why your answer is correct. 

This problem is an example of the distributive property 
that illustrates how to multiply a series of addends 
(in this case x + 5) by the same quantity (3). 3 rows 
of x + 5 is the same as 3 xs + 3 5s, which can be 
written as (3 • x) + (3 • 5) 



© To get ready for science class, a student bought 8 pencils that cost $3 each and 5 rolls of tape 
that also cost $3 each. How much was his total bill? 



Answer the question by using the distributive property, be sure to show all of your work. 

3 (8 + 5) 

= (3 • 8) + (3 • 5) 

= 24 + 15 
= 39 

Total bill = $39.00 



Go to next page. 
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Understanding the Results 2 



Check Your 
Understanding 



0 A student drew these two diagrams to show how the distributive property works. Write a number 
sentence that goes with these diagrams. Be sure your number sentence shows the distributive 
property. 



4cm 



= 4cm 



4cm 



5cm 



3cm 



5cm 



3 cm 



4 (5 + 3) = (4 • 5) + ( 4 • 3) 



© Write a number sentence that shows the distributive property. Draw a 
picture to shows what your number sentence represents. Explain how your 
picture represents the number sentence. 



3(2 + 1) = (3 • 2) + (3 • 1) 



• • 
• • 
• • 



+ • 

1 



This problem is an example of the distributive property that illustrates 
how to multiply a series of addends (in this case 2 + 1) by the same 
quantity (3). 3 rows of 2 + 1 is the same as 3 2s + 3 Is, which can be 
written as (3 • 2) + (3 • 1) 
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Understanding the Results 2 



Use the following chart to record the number of incorrect 
responses to each problem in Checks for Understanding 2: 





Problem 


Incorrect 

Tally 


Total 


1 


7 (4 + 2)= *4 + 7*2 






2 


4 (y + 2) For this problem write only the first step 
that you would write if you were simplifying this 
problem. 






3 


Show how you can use the distributive property to 
rewrite this equation: 3 (2 + 4) = 






4 


6(3+1) = 6*3 Q 6*1 






5 


3 (x + 5) = (Q • x) + (3*5) 

Using what you know about mathematical prin- 
ciples, explain why your answer is correct. 






6 


A student bought 8 pencils that cost $3 each and 5 
rolls of tape that also cost $3 each. How much was 
his total bill? 






7 


Write a number sentence that goes with these 
diagrams. Be sure your number sentence shows the 
distributive property. 






8 


Write a number sentence that shows the distributive 
property. 







Examine the total incorrect for each problem to see which concepts 
your students have the most difficulty with. The following pages 
provide more detailed information about these problems that may 
help inform your subsequent teaching. Some of this information may 
have been covered in your initial professional development session 
and is provided here by way of review. 
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Understanding the Results 2 



We have included, where appropriate, some results from our pilot 
testing of these items in the 2006-2007 school year. These results can 
provide useful information on patterns of responses we have observed 
in a sample of over 2000 students. Clearly, there will be some students 
for whom these Checks for Understanding pose no problems. Others 
may have more difficulty. The specific information on each item may 
help you focus your instruction on distributive property. 
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Understanding and Using the 
Results of Checks for Understanding 2 



The lessons you will be going over with your students in the next 
few days explore content centered on the big idea of the distributive 
property. You will use the ideas underlying the distributive property to 
help students develop a deeper understanding of how the property can 
be used to both simplify expressions and equations and solve other 
types of problems. 

The first Checks for Understanding (given before any instruction) 
focuses on students’ level of understanding of using and explaining 
the distributive property. 

Item 1 assesses students’ understanding as to which term is being 
distributed across the terms in the parentheses. During pilot testing, 
69% of students correctly answered this with a 7. Other errors - 
although only occurring in around 5% of students each - were 2 or 42. 
Clearly, the student who answered 42 does not understand the purpose 
of this type of problem and is simply solving the left hand side of 
the equation and placing the answer in the box. Experience tells us 
that once students have some experience with problems set up in the 
way this one is, we see a decrease in the tendency to simply solve the 
problem. 

Item 2 asks students to provide the first step they would write when 
solving this problem. They are not asked to solve the problem, but 
give the first step. The approach of asking students for the next step 
they would use in solving a problem comes from several research 
studies focused on effectively accessing what students know in a 
domain. Studies have found that this technique is highly predictive 
of performance on other tasks where students were asked to solve a 
whole problem. These results suggest that a “next step” approach may 
be an efficient assessment tool. 

Item 3 assesses students’ ability to solve a distribution problem given 
no scaffolding at all. 
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Understanding and Using the 
Results of Checks for Understanding 2 



Item 4 assesses students’ understanding as to which operation is 
missing in the equation. This item is similar to one we used in pilot 
testing. Students correctly responded with a + sign in over 90% of 
cases. The most common incorrect answer was a multiplication sign. 

Item 5 assesses students’ ability to understand which term is being 
distributed across the terms in the parentheses, this time with a variable 
as part of the problem. Students are also asked to explain why their 
answer is correct. 

Item 6 is a word problem that asks students to answer the question 
posed in the problem by using the distributive property. By explaining 
the process, the students demonstrate that they are not simply 
remembering the process, but are also understanding it as well. 

Item 7 assesses the students’ abilities to evaluate the application of the 
process, demonstrating a yet deeper understanding. 

Item 8 asks students to write a number sentence that shows the 
distributive property and to draw a picture that shows what the number 
sentence represents. This is followed by an explanation. 
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Lesson 1 



I 




Lesson t 



Have you heard of the distributive property? Can anyone 
say how it works? 



show >►>►>►>►>►>►>►>► Overhead 1 



Distributive Property: 

2 (3+4) 

= (2 • 3) + (2 • 4) 

= 6 + 8 
= 14 



This equation shows how the distributive property 
sa Y works. 2(3 + 4) = 2 • 3 + 2 • 4 



show Overhead 2 



The distributive property states 
that you multiply both (or each of the) 
numbers inside the parentheses by 
the number outside the parentheses 
then add (or subtract) the products. 
We can use pictures to see why the 
distributive property works. 



The distributive property states that you multiply both 
(or each of the) numbers inside the parentheses by the 
number outside the parentheses then add (or subtract) 
the products. We can use pictures to see why the 
distributive property works. 



draw 



3 


4 


2 mmm 




^ ### 


#### 



Besides counting, how can we find the number of 
dots on the board? We could do it two ways. We could 
say add the 3 and the 4 across the top to get the number of 
columns in this diagram. 
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Lesson t 



write 



3 + 4 



Then we could multiply this by how many rows there are. 
How many rows are there? 

Right, 2. So we can write 2(3 + 4) 



write 



2(3 + 4) 



What does this equal? Yes, 1 4. How did you get that? 

Yes, you can use order of operations. Add 3 and 4 to get 7, 
then multiply that by 2, and you have 1 4. 



Now let's see another way you can figure out the number 
of dots. You could find out how many dots there are in 
sa Y each part of Figure 1 , then add them together. So, 2 • 3 
for the left part, and 2 • 4 for the right part. 



write 



2*3 2*4 



2 • 3 = 6 and 2 • 4 = 8, so we have 1 4 when we add 
them together. That is the same number we found before. 
sa Y So at first we did this: 2(3 + 4), then we did this 2*3 + 
2 • 4 and now we see that they are equal. 



write 



2(3 + 4) = 2 • 3 + 2 • 4 



These pictures show you that you can separate the dots 
or put them together, and the number stays the same. 
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Lesson 1 



say 



say 



write 



say 



write 



say 



show 



If necessary, repeat this procedure with different dot 
diagrams, such as 3(4 + 1). 

Let's look at another example. 



The distributive property shows us how to multiply a 
series of numbers being added by the same quantity. 
Let's see how it works and how we can use it. 



5(17) 



One way to figure out the answer here would be to use 
mental math and split 17 apart. We can break 17 into 
10 + 7. 



5 (10 + 7) 



We can solve this type of problem using order of 
operations. But we can also use an area model to show 
how the distributive property works. 



>>>>>>>> Overhead 3 
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Lesson 1 



say 



say 



show 



show 



write 



say 



show 



How can we find the area of the large rectangle without 
counting the squares? 



We know we can break up 17 into 10 + 7. 



>>>>>>■>■> Overhead 4 



Make sure students understand that breaking up the 17 
does not change the area because 10 + 7 = 17. Also be 
sure the students recognize that the pictures are the same 
rectangle drawn in different ways. 



>>>>>>>> Overhead 5 



5 (10 + 7) = 5 *10 + 5 *7 



We can show, using these models, that the distributive 
property shows that multiplication is repeated addition of 
the same thing. In this case (10 + 7) • 5 is the same as 
(10 + 7) + (10 + 7) + (10 + 7) + (10 + 7) + (10 + 7) 



>>>>>>>> Overhead 6 
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Lesson 1 



say 

say 

say 

write 

say 

write 

say 

write 

say 

write 

say 



As you can see we can break up the model into 5 sets 
of 10 + 7 which can be rewritten as 5(10 + 7). 



Let's look at another example. 



Do you think the distributive property works with large 
numbers? 



8 ( 351 ) 



How could we use the distributive property to help solve 
this problem? 



8(300 + 50 + 1 ) 



And so we can use distribution to rewrite this as: 



(8 • 300 ) + (8 • 50 ) + ( 8 • 1 ) = 2400 + 400 + 8 = 2808 



Do you think we could use the distributive property to 
help us solve 8 • 99? 

Ask students for ideas. 



8 ( 100 - 1 ) = (8 • 100 ) -(8 * 1 ) = 792 



This strategy will help us a great deal with mental math. 
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Lesson 1 



say 



write 



draw 



say 



say 



write 



All of the above problems use the distributive property. 
How can we write a general formula for the distributive 
property? 




Can you write an expression that shows how you find 
the area of the rectangle? 

Guide students to discovering the response of 2(12 + b) 

How can we use this picture to explain the distributive 
property? Help me explain each step as we do them. 



2(12 + b) = 

Along with your students continue until the expression is 
distributed in the following manner: 

2(12 + b) = (2* 12) + (2 - b) 

= 24 + 2b 
= 24 + 2b 

Create and discuss additional examples if necessary. 
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Lesson 1 



Administer the second 
Check Your Understanding assessment. 
See directions on page 26. 
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Administering 
Checks for 
Understanding 2a 




Administering Checks for Understanding 2a 



Please distribute the Check Your Understanding booklets 
to the students in your class in the order you found them 
in the envelope. Place any unused booklets back in the 
original envelope. 

Today we are going to do some short problems that will 
take about 1 5 minutes for you to complete. Some problems 
may ask you to figure out what the missing numbers are, and 
some may ask you to explain your thinking. 

say 

Read each problem and write the answer that you think is 
best. If there are any words that you are confused about, 
circle them and put a question mark. 

^ These problems are intended to last no more than 15 
minutes. Please stop your students after 15 minutes, even 
if they are not finished. 

^ When the students are working on the problems, if they 
ask a question, please make a note of students’ questions 
on the pages labeled Teacher Notes of Student Questions. 
Please do not provide information to a student that would 
help him or her answer a problem. 

^ In the CRESST folder, you will find yellow copies of the 
Check Your Understanding forms your students are taking. 
If you have suggestions for changes, please write them 
directly on these forms. 

After students have worked on the problems for 15 minutes, 
please collect and score the assessments, and then put all 
of the students’ booklets back in their original envelope. 
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Understanding 
the Results-Checks 
for Understanding 2a 




Understanding the Results 2a 



Checks for Understanding 2a is based on the material covered 
in Lesson 1 on the distributive property. This includes using the 
distributive property to simplify expressions and solve problems 
The following pages contain the answer key for this assessment. 



Check Your 
Understanding 



Answer all questions below. Be sure to show all your work. 

08 (4 + 2)= |8 *4 + 8*2 



© 6(3 + 1) = 6 Q 3 + 6*1 



Go to next page. 
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Understanding the Results 2a 



Check Your 
Understanding 



© Show how you can use the distributive property to rewrite this equation: 
5(3 +2)= | 5 • 3 + 5*2 



Q Write a number sentence that shows the distributive property. Draw a 
picture to show what your number sentence represents. Explain how your 
picture represents the number sentence. 

3 (2 + 1) = (3 • 2) + (3 • 1) 



• • • 

3 • • + • 

• • • 

2 1 

This problem is an example of the distributive property that illustrates 
how to multiply a series of addends (in this case 2 + 1 ) by the same 
quantity (3). 3 rows of 2 + 1 is the same as 3 2s + 3 Is, which can be 
written as (3 • 2) + (3 • 1) 
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Understanding the Results 2a 



Use the following chart to record the number of incorrect 
responses to each problem in Checks for Understanding 2a: 





Problem 


Incorrect 

Tally 


Total 


1 


8 (3 + 2)= *3 + 8*2 






2 


6(3+ 1) = 6 Q 3 + 6*1 






3 


Show how you can use the distributive 
property to rewrite this equation: 

5(3 + 2) = 






4 


Write a number sentence that shows the 
distributive property. 







Examine the total incorrect for each problem to see which 
concepts and problems need to be reviewed and/or re-taught. 

If you have many students who did not understand some of 
the concepts in Lesson l, you may choose to go back over the 
relevant ideas in Lesson 1. While you are doing this, if you have 
students who mastered everything in Lesson 1, you can have 
them go over Lesson 2 in pairs and fill in the worksheet. 
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Understanding and Using the 
Results of Checks for Understanding 2a 



Checks for Understanding 2a focuses directly on the concepts pre- 
sented in the first instructional unit— how the distributive property 
is represented visually and how the distributive property can be used 
to simplify expressions and help solve problems. The key here is to 
have multiple representations available to students. Our work with 
teachers suggests that many students leam the distributive property 
as a rote procedure, and that they simply perform the procedure 
mechanically when they determine that it is needed. When students 
are asked to define the distributive property, they often can quickly 
recite “a(b+c) = ab+ac” from memory. However, when asked to 
demonstrate understanding of the property, such as in problems us- 
ing area models, it becomes clear that many do not understand the 
underlying logic. 

This unit transitions from an area model using squares, to a model 
with a variable. It is critical that students understand that multipli- 
cation can be distributed across multiple terms, even though their 
inclination may be to use the order of operations in a problem with 
no variables. 

The items in Checks for Understanding 2ci are comparable to items 
1,3,4 and 8 on the Checks for Understanding 2. You can use the 
data from Checks for Understanding 2a to determine how well 
students grasped the ideas in Lesson 1 . You can also look at the 
descriptions of the items on pages 15-16 to determine exactly which 
concepts and principles your students may be having problems with 
(if any). 
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Lesson 2 




Lesson 2 



Lesson 2 is designed to be used either by individual 
students, groups, or as a lesson taught by you. It is up to 
you to decide how to use Lesson 2. 



In the last lesson we talked about why the distributive 
property allows you to multiply both (or each of the) 
numbers inside the parentheses by the number outside 
the parentheses then add (or subtract) the products. 



Hand out the Lesson 2 worksheets to your students. As 
mentioned earlier, the way you decide to organize this 
lesson will be determined by how well your students did 
on the first assessment. 

The following pages contain teacher versions of the 
worksheets, which include the expected responses for 
each question. 
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Lesson 2 



Lesson 2 



Distributive Property 



We can use the distributive property to help us solve 
problems like this: 

3(5 + 6) 

We know that because we are multiplying 3 by both 5 and 6 
that we are going to distribute the 3 to the other numbers. 
See if you can complete the problem: 

3(5 + 6) 

= <(>□> + <OA> 

= ( 15 ) + ( 18 ) 



5 or 6 



1) What is the missing number that belongs in the ? 

2) What is the missing number that belongs in the /\ ? 

3a) What is the missing number that belongs in the ^ 



b) Why is the number in the (^) multiplied in both parentheses? 

So it is distributed to both numbers 



4) What is the final answer? 



33 



5) In your own words explain the distributive property. 

Multiply both numbers inside a parentheses by the 
number outside the parentheses then add the products. 
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Lesson 2 




Distributive Property 



A student bought 7 pens that cost $2 each and 4 note 
books that cost $2 each. How much was her total bill? 

6a) Can you use the distributive property to solve this problem? 



6b) If yes, answer the question by using the distributive property, be sure to 
show your work. 



2 (7 + 4) 

= ( 2 • 7) + ( 2 • 4) 
= 14 +8 
= 22 

Total bill = $22 



Let's practice using the distributive property! 



= (3 • 2) + (3 • 4) = (4 • 5) + (4 • 3) = (6 • 1) + (6 • 7) 



7a) 3(2 + 4) 



b) 4(5 + 3) 



c) 6(1 +7) 



= 6 + 12 
= 18 



= 20 + 12 
= 32 



= 6 + 42 
=48 
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Lesson 2 



Lesson 2 



Distributive Property 



Look at this example: 

2(8 + a) 

= ( 2 <3> 8 ) + ( 2 a 
= ( ) + ( ) 



8) What mathematical symbols are missing from the ? X I 

9) What numbers are missing from the blanks? 1 6 and 2a 



10) What is the final answer? 



16 and 2a 



1 1) Using the variables a, b, c, write a general equation for the distributive 
property. 

a (b +c) = (a • b) + (a • c) 

More practice with the distributive property 



12a) 4(x + 3) 



12b) 6(4 + b) 



12c) f(7 + 3) 



= (4 • x) + (4 • 3) 
= 4x + 12 
= 4x + 12 



= (6 • 4) + (6 • b) 
= 24 + 6b 
= 24 + 6b 



= (f • 7) + (f • 3) 
= 7f + 3f 
= lOf 
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Lesson 2 



Administer the third 

Check Your Understanding assessment. 
See directions on page 39. 
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Administering 
Checks for 
Understanding 2b 




Administering Checks for Understanding 2b 



Please distribute the Check Your Understanding booklets 
to the students in your class in the order you found them in 
the envelope. There may be more than one type of form. 
Place any unused booklets back in the original envelope. 

Today we are going to do some short problems that will 
take about 1 5 minutes for you to complete. Some problems 
may ask you to figure out what the missing numbers are, 
and some may ask you to explain your thinking. 

say 

Read each problem and write the answer that you think 
is best. If there are any words that you are confused 
about, circle them and put a question mark next 
to the circle. 

^ These problems are intended to last no more than 15 
minutes. Please stop your students after 15 minutes, even 
if they are not finished. 

^ When the students are working on the problems, if they 
ask a question, please make a note of it on the pages la- 
beled Teacher Notes of Student Questions. Please do not 
provide information to a student that would help him or 
her answer a problem. 

^ In the CRESST folder, you will find yellow copies of the 
Check Your Understanding forms your students are taking. 
If you have suggestions for changes, please write them 
directly on these forms. 

After students have worked on the problems for 15 min- 
utes, please put all of the students’ booklets back in their 
original envelope. 
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Understanding 
the Results-Checks 
for Understanding 2b 




Understanding the Results 2b 

Checks for Understanding 2b is based on the material covered 
in Lesson 2 on the distributive property. This includes using 
the distributive property to simplify expressions and solve 
problems. The following pages contain the answer key for this 
assessment. 



Check Your 
Understanding 




Answer all questions below. Be sure to show all your work. 


O 6 (y + 3) 

For this problem write only the first step that you would write if you were simplifying 
this problem. 


possible answers: 6/ 




(6 • V) + (6 • 3) 




6 • y + 18 




6/ + 18 


© 6(3+ 1) = 6. 3 


+ 6-1 




Go to next page. 
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Understanding the Results 2b 



Check Your 
Understanding 



Complete the number sentence below. 



© 



y(15 +5) = ( 



' 15) + (y • 5) 



Using what you know about mathematical principles, explain why your answer is 
correct. 

This problem is an example of the distributive 
property that illustrates how to multiply a series of 
addends (in this case 15 + 5) by the same quantity 
(y). y rows of 15 + 5 is the same as y 15s + y 5s, 
which can be written as (y • 15) + (y • 5) 



Q Frank was washing cars over the summer. On one day he washed 12 cars in the morning 
and 6 cars in the afternoon. It took him 10 minutes to wash each one. How much time 
did he spend washing cars? 

Answer the question by using the distributive property, be sure to show all of your work. 

10 (12 + 6) 

= 10 • 12 + 10 • 6 
= 120 + 60 
= 180 

= 180 minutes or 3 hours 



Go to next page. 
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Understanding the Results 2b 



Check Your 
Understanding 



© A student drew these two diagrams to show how the distributive property works. 
Write a number sentence that goes with these diagrams. Be sure your number 
sentence shows the distributive property. 



3cm 



3 cm 



+ 3cm 



4cm 2cm 



4cm 



2 cm 



3 (4 + 2) = (3 • 4) + (3 • 2) 
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Understanding the Results 2b 



Use the following chart to record the number of incorrect 
responses to each problem in Checks for Understanding 2b: 





Problem 


Incorrect 

Tally 


Total 


1 


6 (v + 3) 






2 


6 (3+1) = 6* +6*1 






3 


y (15 + 5) = (Q - 15) + CV - 5) 






4 


Answer the question by using the 
distributive property, be sure to show all 
of your work. 






5 


Write a number sentence that goes with 
these diagrams. Be sure your number 
sentence shows the distributive property. 







Examine the total incorrect for each problem. This total will give 
you a sense of how many of your students have mastered the 
concept of the distributive property. 

These concepts will come up again in future units, so students 
will have more opportunities to practice what they have learned. 
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Understanding the Results 2b 



Checks for Understanding 2b is based on the material covered in 
Lesson 2 on the distributive property. This includes solving word 
problems using the distributive property and also provides extra 
practice for students to practice using the property in different 
contexts. 

The items in Checks for Understanding 2b are comparable to 
items 1, 2, 5, 6, and 7 on Checks for Understanding 2 (see page 
9). You can use the data from Checks for Understanding 2b to 
determine how well your students have grasped the ideas in 
Lesson 2. You can also look at the descriptions of items 1, 2, 

5, 6 and 7 on page 15 to determine exactly which concepts and 
principles your students maybe having problems with (if any). 
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Appendix C: 

Sample Alternative Professional Development Materials 




Using Data to Make 
Instructional Decisions 



Terry Vendlinski 
Bryan Hemberg 



Norwalk - La Mirada USD 
October 1 , 2007 



Generating Data 



• Assessment in California Education 

• Entry-level 

• Summative 

• Progress Monitoring 

• Standards and Blueprints 

• California Standards Tests 

• Benchmarks 



Accessing the Data 
(Data Director) 



• Login to Data Director 
(https://www6.achievedata.com/nlmusd) 

• Tabs vs. Menus 

• Assessments 

• Exams 

• Reports 

• Other Tabs 



Refining Assessment Dataset 



• Last year’s 6 th grade CST results for Math 

• Make a report 

• Delete Cluster 6, Test Taken, Test Taken Text, Items 
Attempted, and Include Indicator. 

• Modify columns so we can sort on grade, period and teacher 
(first and last name). 

• Refine data set to include only your first period (even if not 
period one) 

• Choose Summary to type summary and include graph (color 
or grayscale) of proficiency level 



Cluster Scores (handmade) 



• Does one cluster better predict overall 
performance? (Sort by each cluster score) 

• Will weighting make a difference? 

• What does each cluster mean ? 

• How many items tested in each cluster? 

• Other explanations for variability in student 
CST performance? 

• Can this focus instructional improvement? 



Cluster Scores (pre-made) 



• Click on Report tab and choose CST Cluster 
Scores 

• Modify data set for your first period in 2006 - 
2007 

• Sort by cluster 

• Helpful in making instructional decisions? 

• Helpful in focusing on specific students? 



Cluster Scores (modifying) 



• Make a duplicate report, provide summary 
and save 

• Modify column to add 2005 - 2006 Raw 
Cluster Scores. 

• Reorder scores so ’06 and ’07 Clusters are 
side by side 

• Are 5 th and 6 th grade clusters associated? 

• Did same clusters result in same ordering? 



Cluster Scores (Pivot Table) 



• Click on Report Tab and choose Pivot Table 

• Use 2005 - 2006 as first data set 

• Use 2006 - 2007 as second data set 

• Use linear scaling 

• What do circles mean? 

• Choose one group above and one group 
below diagonal. 



Summative Conclusions 



• What might summative data tell you? 

• What are limitations of the data we analyzed? 

• Data Set limitations (what data is available) 

• Analytical limitations (what tools are available) 

• Administration limitations (when data is available) 



Refining Examination Dataset 



• Final ALS Benchmark 2 for 6 th grade Math in 
2006 - 2007 

• View Classroom Performance Summary 
menu item for your first period. 

• Sort by clusters 

• Would you identify the same cluster as most 
important instructional focus? 



Examination Details 



• Examine mean, median and range in Test 
Statistics. 

• Meaning of quartiles 

• Meaning of mean and standard deviation 

• Confidence intervals 

• Reliability measures 



Examination Item Details 



• Item Statistics 

• P-values 

• Point Biserial Statistic 

• Item Analysis 

• Distractors (Q5, Q18, Q24) 

• Differential Item Functioning 



Conclusions from Progress 
Monitoring 



• What might progress data tell you? 

• What are limitations of the data we analyzed? 

• Technical Quality limitations ? 

• Administration limitations? 

• Correlation with summative measures? 



Implications 



• What type of data did you find most useful? 

• What is your purpose ? 

• How will you use the data ? 

• Proposal for future meetings 

• Benchmark analysis 

• Implications for instruction (TLCs) 






Terry Vendlinski - vendlins@ucla.edu 



Bryan Hemberg - hemberg@cse.ucla.edu 



Self-Regulated Success 



Metacognitive Strategies toward 
Improved Academic Achievement 



Keith Howard, Jessica Ulloa 




Roadmap 



1 . Self-Regulated Learning 

• Students taking responsibility for learning 

• Self-directed behavior & metacognition 

2. Self-Efficacy & Goals 

• Student beliefs and their impact 

• Mastery vs. Performance Goals 

3. Attributions & Affect 

• Attributing causes for successes 

• Anxiety & stereotypical beliefs 

4. Teacher & Classroom Influences 

• Rewards 

• Feedback 

• Expectations 




Present 




Advance Organizer 

1. Feedback 

• Calendar, Structured Worksheet, etc. 

2. Math Class Vignette 

• Building Self-Efficacy 

3. Self-Efficacy 

• Sources 

• How Instruction Can Influence 

4. Mastery vs. Performance Goals 

• Defined 

• Social Comparisons 

• Views on Ability 




Self-efficacy 

• “Beliefs in one’s capabilities to 
organize and execute the courses 
of action required to produce 
given attainments” (Bandura, 1997:2003) 

• Cognitive Appraisal of Capabilities 

• Context or Domain Specific 

• Affects future actions (i.e. choice 
of tasks, persistence, effort) 



Self-Efficacy & Academic Motivation 

Self-efficacious students: 

• Take on difficult and challenging task more readily than 
do inefficacious students 

• Exert more mental effort in academic tasks 

• Demonstrate increased persistence 

• Exhibit less anxiety 

• Demonstrate better self-regulation of learning through 
goal setting, monitoring, self-evaluation, and strategy 
use 



(Zimmerman, B. J. 2000) 



Self-Efficacy Sources 

• Vicarious experiences can be very 
influential if student identifies with model 

• Social persuasion can influence, but this 
influence is not enduring 

• Mastery experiences are the most 
powerful influences 



(Woolfolk, A. 2007) 




Self-Efficacy for Specific Tasks 



• Solve for x: 2 + x = 5 

• Solve for x: 2x = 1 0 

• Solve for x: 1 2 = x - 1 2 2 

• Evaluate: 5x7-6 x2 + 3 2 

• Add: (9x 3 + 12x) + (16x 3 - 4x + 2) 

1 

• Rewrite with positive exponents: ~ 8 -5 

2.x y 



Information Processing Model 



METACOGNITION JHHH 
rehearsal 

Response 

y .. 

LONG- 
TERM 
MEMORY 





Forgotten 

(perhaps 

recoverable) 




f ft 



Information Processing Model 



OTIKill II I 




attention perception 



METACOGNITION 



SENSORY 

memory m 



Response 



rehearsal 



encoding 



WORKING 

MEMORY 








LONG- 

TERM 

MEMORY 




(Eggen & Kauchak, 2001, p. 261) 




• Virtually unlimited capacity 

• Permanently stored 

• Composed of networks and schemas 

• Information encoded from working 
memory 




Transition to Long-Term Memory 



• Organization 

• Knowledge structures (schemas) 

• Elaboration 

• Increase meaningful connections in context 

• Rehearsal 

• Maintain information in working memory long enough 
to allow linking to meaning in long-term memory 

• Automaticity 

• Frees up processing resources 



Automaticity 



What types of knowledge should be 
automated? 

Advantages of Automation? 

Long-Term Working Memory? 

Process of Automation: 

• Cognitive Stage - thinking about each step 

• Associative Stage - chunking into larger units 

• Autonomous -procedure requires little conscious 
processing 



Goal Theory 



• Mastery Goal - learning for 
learning’s sake; a desire for 
self improvement, 
irrespective of the 
performance of others 




• Performance Goal - 
focuses on competition and 
social comparisons; the 
main objective is to 
outperform others 






Mastery vs. Performance Goals 



• Mastery Goal 

• Positive attitude toward task 

• Self-monitoring 

• Makes connections with prior learning 

• Performance Goal 

• Focus on memorization/rehearsal strategies 

• Less engaged 

• More focused on getting done than learning 



Different Views of 
Performance: Goal Orientation 





Mastery 

Orientation 


Performance 

Orientation 


Success 


Improvement, 


High grades, high 


defined as... 


progress, mastery, 
innovation 


performance compared 
with others 


Error viewed 


Part of the learning 


Failure, evidence of lack 


dS ■ ■ ■ 


process, informational 


of ability 


Ability viewed 

dS ■ ■ ■ 


Developing through 
effort 


Fixed 



Performance Approach & 
Avoidance 



Performance Approach - desire to look 
favorable in comparison to others 

Performance Avoidance - desire to avoid 
looking bad in comparison to others 

Performance goals are encouraged by 
school practices such as posting of grades 
grading on curves, and norm-referenced 
tests 



Social Comparison 

• Comparison information can 
enhance students’ 
motivation, but not 
necessarily their self- 
efficacy or learning 

• The perception that one is 
improving is hypothesized to 
raise self-efficacy 

• The posting and public 
celebration of grades can 
encourage performance 
goals if learning is not 
emphasized 




Encouraging Mastery Orientation 



• Connect subject matter to 
useful future application in 
life 

• Emphasize understanding 
more than grades 

• Suggest incremental view 
of intelligence 

• Acknowledge mistake- 
making as part of learning 
process 






Appendix D: 

Examples of Professional Development Website Materials 









Calendar 

Surveys 



Content Map 
Measures 
Message Board 



The purpose of the PowerSource study is to try out new Grade 6 formative 
pre-algebra assessments and associated instructional materials. GRES ST 
researchers propose to determine the effectiveness of our materials., which are 
based on helping teachers to understand whether students are learning the 
fundamental mathematical principles underlying mastery in aigebra, This 
project is designed to develop assessment arid instructional materials that can 
be used throughout the school year to evaluate student understanding of 
foundational concepts in pre-algebra and algebra. The topics covered in this 
material are likely to be covered in tests required by the State. 

Over the course of the school year., PowerSource study participants will be 
introduced to the four Big Ideas found below: Rational Number Equivalence ., 
Properties of Arithmetic., Solving Equations., and Review and Applications. 

On the menu to the left you will find resources related to the overall scope of 
the study., including a calendar., surveys., and a content map which visually 
connects the Big Ideas to Algebra I. 

Within each of the Big Ideas sections found below are resources that will aid 
and assist participating Powers ounce teachers. Descriptions of the units., study 
materials., research links., lesson explanations., and frequently asked questions 
are just some of the many useful items provided. 

You will be able to access each section after it is presented to you at the 
Professional Development Institutions that preface each unit. 



Rational Number Equivalence 



Properties of Arithmetic 



Solving Equations 









j r 



r 




Properties of Arithmetic 



< back 



Oveiview 

FAQs ► 

Big Ideas 

Content Map 
Explanation 

Connections 

Teaching Resources 

Misconceptions 
Teaching Aids 
Research Links 
Results 

Teacher Handbook 

Using die Checks 
What You'll Need 
Scoring 

Next Steps 



1. When will my students ever use this? 

* Your students probably already use the distributive property when 
multiplying multi pie -digit numbers by other non-zero numbers.; however, 
they probably aren't aware that is what they are doing, We will use their 
experience of multiplying such numbers as a beginning point in our 
professional development and a beginning point in instruction, 

* In the Properties of Arithmetic unit., students will apply the concept of 
multiplication to develop an understanding of the distributive property and 
adding "'like terms." 

* Eventually., students Will use this concept to solve linear and quadratic 
equations, and to factor expressions so they can simplify rational 
expressions (these topics Will appear in later grades), 



2. My students are confused as to what an operator is. 

■ An operator, as used in the Checks for Understanding materials, is 
referring to one of the basic mathematical operations; addition (+), 
subtraction (-), multiplication ( ■ ), or division (•f); 



3. Why do we have to use the distributive property when the order of 
operations can get the same answer? 

* At times it is faster and easier to use the distributive property , For 
example, when multiplying 37 times four mentally, it is probably easier to 
think 30 times 4 is 120 and 7 times 4 is 26, and then add 120 and 28 to 
determine the final product is 148. This same principle can be used when 
trying to calculate a 15% tip. It is often easier to calculate 10% of the bill, 
then take ha if of that (5%) and add the two results, rather than multiplying 
fifteen times the total. 

* As students begin to study algebra and pre-algebra, they will encounter 
situations when they can't combine terms inside the parentheses (for 
example [x + 2]), but must multiply the quantity in order to easily solve 
the problem. The ability to distribute in such situations will become even 
more important when students try to solve equations like 

(x r 2) (x + 3) = 7. 

* Non-graphical solutions to such an equation (such as factoring, completing 
the square or the quadratic formula) will require students to understand 
and use distribution if they are to comprehend, rather than just 
algorithmically manipulate (e.g,, F.O.I.L.) a solution strategy. 

* Order of operations is not always obvious, such as when you multiply 

2 ■ 2 34. In this case you are actually adding 2 % 4 2 -& or adding two 2s 
and two t4s or adding 2 ■ 2 and 2 ■ (distribution). 



4. My students are not used to the formatting of the questions, what 
should I do? 

* The construction of the questions in the Check for Understanding forms 
follow a few different formats, There are "'basic" items that require 
students to fill in the blank or provide a solution to a problem, and 
"explanation" problems that require students to write out an explanation 
for their solution, sometimes in conjunction with mathematical work 
associated with the item and/or previous items, 

* Other types of problems are ones where students just have to complete 
the first step, The approach of asking students for the next step they 
would use in solving a problem comes from several research studies 
focused on effectively assessing what students know in a domain. Studies 
have found that this technique is highly predictive of performance on other 
tasks where students were asked to solve a whole problem, These results 
suggest that a "next step" approach may be an efficient assessment tool. 
This type of item is usually the most unfamiliar to students. Be sure to 
remind students that, although there may be multiple steps within a single 
problem, they only need to write in the first step they would do— they don't 
need to solve the whole problem. 

* It is also important to have students carefully read the instructions that 
precede each item, so that they know what is being asked and what type 
of response is expected. You may also highlight or explain to students 
what the problem is asking them to do without actually answering the 
question for them. 



5. My students are having difficulty understanding the terms contained 
in the questions on the Checks for tfnderstcmding , What can 1 do? 

• To ensure that students are able to understand what is being asked by 
each question, it may be beneficial to ask students if there are any 
mathematical terms contained in the questions that they do not recognize 
or know the definition for. By going through the materials before you 
administer them to students you may be able to pick; cut any terms your 
students may not recognize and introduce them beforehand as part of 
your instruction. 



6. My students are having difficulty providing a written enplanation to 
some of the questions. They know the ansuier, just not how to put it 
into words. 

* Have your students say aloud how they would answer the question, in the 
most accurate and concise way they can. This may take more than one 
attempt. Then have them write the answer they have verbalized. 

■ As part of instruction j have students justify their thinking to a partner by 
explaining the logic they used to solve a question that you pose. This 
justification can be verbal or written. 



7. How con I help my students avoid making careless errors? 

* Students and teachers will make careless errors in their thinking and 
computation. A way to avoid these errors is to ask students to show their 
work. Teachers should also remind students to carefully read the question, 
complete their work slowly, and spend time practicing the type of problem 
they are having difficulty with. 

* Encourage students to check their answers after solving a short answer 
question to ensure their solution works in the problem as it was originally 
presented . 



8. There were many points during the lesson where I could perceive 
that my students were unfamiliar or had forgotten a previous concept. 
Is it ok for me to interrupt the lesson to instruct the students on these 
concepts? 

■ Yes. 
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Overview 

The lessons you wiil be going over with your students in the next couple of days 
cover the big idea of the properties of arithmetic— specifically., that m u iti p I i cati o n 
can be defined as repeated addition of the same quantity; also that multiplying a 
quantity like (x + 2) by 3 illustrates the distributive property, You Will use the 
ideas underlying the distributive property to help students develop a deeper 
understanding of how the property can be used to both simplify expressions and 
equations., and solve other types of problems, Developing an understanding of 
the idea of distribution., and not just the memorization of a process, may help 
students avoid common misconceptions that interfere with understanding the 
distributive property (e.g. incorrect distribution)., and serves as a foundation for 
understanding other concepts (e.g. solving equations., simplifying expressions). 
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A rational number is any real number (negative , positive., or zero) that can 
be expressed as a quotient of two integers, 



° 0 can be expressed as 



O 8 can be expressed as 



I 
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Q -4 can be expressed as 



Q .62 can be expressed as 



hi 

inn 

j I 



o 2' can be expressed as 

4 -l 

• Rational numbers can have several meanings: 

0 Fraction (part-whole): !$> means 3 out of 4 
O Division: % means 3 divided by 4 
0 Ratio (part: part) : -A means 3 compared to 4 
O Measure: -A nrieans three-quarters of one unit 
O Multiplicative operator: 34. means three-quarters of (or times) a 
thing 



Questions to Consider 

* How deep is my student's understanding of the idea that x/x = 1? 

* How are rational nu mi hers used in the curriculum I teach in grade x? 

* How is this intended to enrich or improve my understanding of the content 
and of how to teach the content? 



What "Big Idea” is this designed to address? 

■ Equivalent rational numbers may be created through the application of the 
multiplicative identity (multiplying any fraction by x/x) 

* The multiplicative identity states that if you multiply any number by l., and 
the resulting product is the same number. 

* Any number divided by its own Value., or written over its e if m a fraction, is 
1 (except Ci), 

* If you multiply an;/ number by x/x the resulting product is equivalent to 
the original number 



Teaching Rational Number Equivalence 

■ Students are often taught to find equivalent fractions through the use of a 
procedure, and are not provided with an explanation as to why the 
procedure is mathematically sound. 

* One of the goals of this project is to reduce the numiber of such facts that 
students (and you) need to memorize to a minimum n urn her of simple 
ideas and to build more complex ideas from a few number of simple 
concepts , 

* Using such a structure suggests that students will have to memorize less., 
and you will have to teach fewer ideas in depth and have an opportunity to 
apply them more broadly. 
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Most students struggle for years trying to understand the concept of Rational 




Content Map 


Numbers. In part., this struggle arises because of fhe way the concept is taught, 




Explanation 


We have taken one aspect of Rational Numbers (developing equivalent rational 




expressions) and tied it to a concept that most students understand well (the 




Conner lions 


multiplicative identity), Making such a linkage seems to avoid common 


,fr.' 




m i s u n d e r sta n d i n g s th at i nte rf e re w i th u n d e r sta Tiding R ati o n a 1 N u m b e rs ( e . g . 




Teaching Resources 


"whatever you do to the top., you do to the b otto m . ") , and serves as a foundation 




Misconception? 


for understanding other concepts (e,q. solving equations , simplifying Rational 




Expressions., and operations on Rational Numbers), 




Teaching Aid? 




Research Links 






Rasul ts 
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Scoring 






Next Steps 


. 
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Overview 

The lessons; you wifi be going over with your students in the next few d a y s 
explore content centered on the big idee of solving equations. In this unit., 
students will apply the concept of equality to solve one- step equations for an 
unknown. You will use the ideas underlying inverse operations and the additive 
identity property to help students develop a deeper understanding of how to 
solve equations m order to find an unknown. Eventually., students will use this 
concept to substitute., simplify., rewrite and otherwise transform a variety of 
equations (e.g. linear, quadratic., exponential, and rational) into a form that they 
easily solve, In each case, the understanding of why one expression is equivalent 
to its transformation is essential to solving the presented problem. 




Solving Equations 

Overview 



FAQs 


What are common errors students make when asked to find equivalent 


Big Ideas 


fractions, and why do they make these errors? 


Content Map 


1. Misconception: '''Whatever you do to one side., do to the other" 


Explanation 


This statement is mathematically incorrect, Students are often taught this 


Connections 


adage because it does work with some simple equations involving whole 
numbers., it does not however, work with all equations, It should therefore 


Teaching Resources 


net be taught as a procedure for students to follow- 


Misconceptions ^ 


Strategies: 


Teaching Aids 


Demonstrate why this type of thinking is incorrect, 


Research Links 


For example, when solving the equation — + — = 4, it is permissible to 

3 4 


Results 


H 


Teacher Handbook 


multiply the first term by and the second term by - , but one need not do 
anything to "the other side" of the equation, 


Using the Checks 




What You'll Need 


The quintessential rule when teaching solving equations is: "whatever you 
do to one side., do to the other." Although this may be true for most 


Scoring 


additions, subtractions, multiplications and divisions, it is not always true 
nor is it required in every instance, 


Next Steps 


For example: 

• One cannot divide both sides by zero. 

• Adding or subtracting zero on one side of an equation. 

• Trying to take the absolute value of or squaring both sides of an 
equation could also lead to inaccurate results. 

If asked to solve the equation 3x = 2x, what would you do? 

• If you say it is unsolvahle, look again, In fact, if you try to divide 
both sides by "x," you are left with 3 = 2. Cieariy this cant be true 
and it might reinforce the notion that the equation has no solution, 

• Is it even allowable to divide both sides by x? The adage suggests 
that it is, however, division by x is not allowed if x = 0 (division by 
zero is undefined!). So you can 't "do the same thing to both sides" 
here if x — 0. 

Let's try another way, The equation 3x = 2x can be rewritten as x+x+x = 
x+x because that is w h at multiplication means. Now we can: 

• Subtract an x from both sides (the subtraction property of equality) 
to get x+x = x 

• Subtract another x from both sides (or subtract 2x from both sides 
from the start) to get x = 0 

• At this point, you can see that x = 0, Indeed, if we check our work: 
by substituting 0 for x, we find that 3 * 0 = 2 * 0 since 0 = 0 (the 
identity property), 

'What we encourage teachers to avoid is giving students new rules to 
memorize, especially when the rules might lead to misconceptions, 

2. Misconception: Students see the variable as only representing an 
unknown object 

Strategies: 

A variable can represent any number (a variable) of things, 

Note that a classic barrier to understanding the algebra is that students 
see the variable as only representing an unknown object. 

In reality, a variable need not represent a concrete thing or single 
unknown, They can also represent any number (a variable) of things, 

For example, when asked to represent art increase in the width of a 
square by two units, many students first want to know the original 
dimensions of the square. In reality, since the side of the square could be 
any number (x), the width of the new square could be represented as 
"x+2", 

3. Misconception: Students solving equations through the use of ''''mental 
math" 

This often leads to incorrect solutions, something that can easily be 
avoided through the processes of explanation of work completed and the 
checking of solutions, 

Strategies: 

Explanations help students understand the mathematical principles they 
are using to find their solutions, 

In the discussion of the Solving Equations unit we have used worked 
examples (mathematical proof) and advocate its use in instruction 

• Worked examples (where more and more steps are successively 
removed from the end of a problem) have been demonstrated to 
help deepen student understanding (Sweller & Cooper, 1985), 

• Similarly, asking students to show and explain the mathematical 
''big ideas" that justify each step in a solution, have been shown to 
improve student understanding (Renkl, 2002). 

'While students may be able to solve equations mentally, they should show 
work (and explain steps) to develop understanding, reduce the recall 

mistakes (e.q, inappropriate fact family such as ——A , so x = 3), and 

12 

minimize calculation errors (100 - x = 87; x = 23), 

4. Misconception: Students "cancel out" the variable when solving an 
equation involving multiplication or division, 

This often leads students to believe that x/x becomes 0 instead of 1 since 
it has been "cancelled," 

Strategies: 

It is important to make clear that to "isolate x" when we are multiplying or 
dividing, we need to multiply (or divide) by one (not zero). 

• For this reason, we suggest teachers avoid using the word "cancel" 
when simplifying a rational expression or equation with students. 

• Instead it is probably better to indicate that the coefficient of the 
variable is one. The multiplicative identity then allows us to "ignore" 
the coefficient, 

• Just as the addition of "opposites" produces the identity number for 
addition (0), there is a way to use multiplication to produce the 

i d e ntity n u rn b e r f o r m u ftr p ! i cati o n ( 1 ) . 

Jf 

How would you show that the expression ( w here y =£ 0) can be 

1 

rewritten as a ■ — ?. 

7T ] 

O n e p o s s i b I e " w o rk e d e x a m p I e " d e m o n strati n q 

y y 




Step 1 j Given expression 

—,y * o 
y 

Step 2 y- Multiplicative identity. The value of a quantity 

— *1 doesn't change when multiplied by 1 so this is 

y equj v a lent to th e a b o v e . 

Step 3 1 Any non-zero number divided by itself is one, 

— Since y is not zero, 1/y is a real number and is 

x m y non-zero 

x I 

y 

Step 4 \ A number times its reciprocal is 1, 

T • — 

y 

i 

Step 5 1 Division by 1 does not change the value of the 

T • — dividend. 

y 
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Background Survey 



This short survey will provide us with information regarding the instructors that 
are participating in the study. The answers to these questions will give insight 
into teacher's attitudes towards a s s e s s m e nts ., teaching and educational 
experience., and technology in the classroom, Please answer each question 
and return with your first set of materials. It should only take about TO 
minutes to complete the survey, 

To do w load the survey t click bene 



Implementation Surveys 

This short survey will provide us with information regarding how the instructors 
that are participating in the study chose to implement the PowerSource 
materials. This answers to these questions will give insight into what Checks for 
Understanding forms were i j s e d , the functionality of the Teacher Handbook., and 
how the Lesson 2 Worksheet was implemiented, Please answer each 
question and return with each set of materials. It should only take about 
TO minutes to complete each survey. 

Dowioad the Rational Number Equivalence survey here 

Dowload the Properties of Arithmetic survey here 

Dowioad the Solving Equations survey here 

Dowload the Review and Applications survey here 
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ITEM 


Item Text 


1 


What do you need to add to eighty-three to make one hundred? 


2 


Write the fraction 3/9 on your answer sheet in its simplest form. 


3 


There were two thousand people at a concert. Nine hundred and 
ninety-two of them 

were women. How many of the people were not women? 


4 


Write a fraction that is less than 4/9 . 


5 


Write a different fraction that is equivalent to three-fifths. 


6 


b = 14 + a. When a equals 7, what is the value of b? 


7 


If 12n = 3621 , then n equals: 

a) 3 

b) 7 

c) 36 

d) 63 


8 


Which of the following ratios is equivalent to the ratio of 6 to 4? 

a) 12 to 18 

b) 12 to 8 

c) 8 to 6 

d) 4 to 6 

e) 2 to 3 


9 


For all numbers k, 
k + k + k + k + k can be written as 

a) k + 5 

b) 5k 

c) k5 

d) 5 (k + 1) 


10 




11 




12 




13 




14 




15 




16 


1 6. Which of the following is equal to 6 (x + 6) ? 

a) x + 12 

b) 6x + 6 

c) 6x+ 12 

d) 6x + 36 

e) 6x + 66 


17 


Simplify using the distributive property. 

y(y-6)= 
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How much change will John get back from $5.00 if he buys 2 
notebooks that cost $1 .80 each? 

a) $1.40 

b) $2.40 

c) $3.20 

d) $3.60 

The perimeter of a square is 36 inches. What is the length of one side 
of the square? 

a) 4 inches 

b) 6 inches 

c) 9 inches 

d) 18 inches 

Which of the following numerical expressions gives the area of the 
rectangle below? 

a) 4 • 6 

b) 4 + 6 

c) 2 (4 • 6) 

d) 2 (4 + 6) 

e) 4 + 6 + 4 + 6 

What is the value of x in the triangle? 

a) 65° 

b) 82° 

c) 90° 

d) 92° 

e) 98° 



24 




25 


If 3 + w = b, then w = 

a) 39 

b) b • 3 

c) b + 3 

d) 3 — b 

e) b - 3 


26 




27 




28 




29 




30 




31 


In which list of fractions are all of the fractions equivalent? 

a) 12 , 24 , 46 

b) 23 , 46 , 812 

c) 25 , 410 , 850 

d) 34 , 46 , 68 



18 



19 



20 



21 



22 

23 
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32 


n is a number. When n is multiplied by 7, and 6 is then added, the 
result is 41. 

Which of these equations represents this relation? 

a) 7n + 6 = 41 

b) 7n + -6 = 41 

c) 7n • 6 = 41 

d) 7(n + 6) = 41 


33 


The diagram shows triangle PQR. 

Work out the sizes of angles a, b, and c. 




What would be your answer if you were asked to multiply 8 • (x + 34) ? 

a) 8x + 34 

b) 8 34x 

c) 8x + 6 

d) x + 6 




Write a fraction that has a denominator of 100 and is equivalent to 7/20 . 




Explain why the fraction 1/2/3/4 is equivalent to the fraction 2/3? 




What value of x makes the equation true? 
x-9 = 32 

a) 23 

b) 41 

c) 32 
HI Q 




Solve: 6n = 36 

a) 12 

b) 2 

c) 30 
HI 6 




What is the value of p in the equation below ? 
14p = 4 

a) P = 4 

b) p = 16 

c) p = 4 14 

r\ n = 3 3 A 




What is the next step to solve this equation? 
x - 7 = 13 

a) Subtract 7 from both sides 

b) Add x to both sides 

c) Add 7 to both sides 

rh <si ihtrart 1 3 from hnth cirlpc 




Sam's uncle is 21 years older than Sam. His uncle is 42. What equation could 
you use to solve 
for Sam's age, s ? 

a) s + 21 = 42 

b) 4221 = s 

c) s - 21 = 42 

H\ c- _ A1 — 01 




Which of the following shows the distributive property being used correctly 
to simplify the 
expression: 3(4) + 3(2) 

a) 3(4)(2) 

b) 3(4 + 2) 

c) 4(3 + 2) 

rl\ /W3\ ^ 
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SOURCE 


CA Standard 


QCA, key stage 3, p. 4, #2 


NS 2.0 


QCA, key stage 3, p. 4, #7 


NS 2.4 


QCA, key stage 3, p. 5, #1 9 


NS 2.0; NS 2.3 


TIMSS, grade 8, 2003, item number: m022012 


NS 1.1 


QCA, key stage 3, p. 8, #5 


NS 2.1 
AF 1.2 


QCA, key stage 3, p. 4, #3 






NS 1.3 


NAEP, grade 8, 2003, #58 


NS 1.2 


TIMSS, grade 8, 1999, item number: pi 1 


AF 1.0 


QCA, key stage 3, tier 4-6, paper 2, p. 27, #26 




TIMSS, grade 8, 2003, item number: m032036 




QCA, key stage 3, p. 8, #1 3 




QCA, key stage 3, tier 4-6, paper 2, p. 27, #26 




QCA, key stage 3, tier 6-8, paper 2, p. 1 1 , #1 0 




QCA, key stage 3, tier 6-8, paper 2, p. 1 1 , #1 0 




NAEP, grade 8, 2005, #41 


AF 1.3 



QCA, key stage 3, tier 6-8, paper 2, p. 15, #14 AF 1.3 
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NAEP, grade 8, 2003, #7 


NS 2.3 


NAEP, grade 8, 2003, #10 


AF 3.1 


NAEP, grade 8, 2003, #34 


AF 3.1 


NAEP, grade 8, 2003, #32 


MG 2.2 


QCA, key stage 3, p. 8, #1 1 


MG 2.2 


QCA, key stage 3, tier 3-5, paper 1 , p. 21 , #21 
Also in QCA, key stage 3, tier 4-6, paper 1 , p. 
15, #14 




QCA, key stage 3, tier 3-5, paper 1 , p. 21 , #21 
Also in QCA , key stage 3, tier 4-6, paper 1 , p. 
15, #14 




NAEP, grade 8, 2003, #47 


MR 3.3 


QCA, key stage 3, tier 4-6, paper 2, p. 27, #25 




QCA, key stage 3, tier 4-6, paper 1 , p. 20, #21 




TIMSS, grade 8, 1999, item number: LI 7 




QCA, key stage 3, tier 6-8, paper 2, p. 15, #14 




TIMSS, grade 8, 2003, item number: m022253 


AF 1.3 


TIMSS, grade 8, 1999, item number: N14 


NS 1.1 
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TIMSS, grade 8, 1999, item number: B12 


AF 1.0 


QCA, key stage 3, tier 3-5, paper 1, p. 20, #20 


MG 2.2 


Adapted from PISA item 




Adapted from PISA item 




Adapted from PISA item 




Adapted from 6th Grade Benchmark Test~3~ 
Norwalk La Mirada--item 10 


AF: 1.1 


Adapted from 6th Grade Benchmark Test-3~ 
Norwalk La Mirada--item 11 


AF: 1.1 


Adapted from 6th Grade Benchmark Test-3- 
Norwalk La Mirada-item 12 


AF: 1.1 


Adapted from 6th Grade Benchmark Test-3~ 
Norwalk La Mirada-item 14 


AF: 1.1 


Adapted from 6th Grade Benchmark Test--3~ 
Norwalk La Mirada-item 16 


AF: 1.1 


Adapted from 6th Grade Benchmark Test— 3— 
Norwalk La Mirada-item 24 


AF: 1.3 
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Adapted from 6th Grade Benchmark Test-3 
Norwalk La Mirada-item 28 



- AF: 2.3 
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6th Grade Standards Domain Checks for Understanding Item Numb 



RNE SE RA 

Number Sense 

1.0 Students compare and order positive and negative fractions, decimals, and RNE, SE, RA RN-EX-12 

mixed numbers. Students solve problems involving fractions, ratios, RN-EX-15 

proportions, and percentages: 

1.1 Compare and order positive and negative fractions, decimals, and mixed RNE, RA 

numbers and place them on a number line. 

1.2 Interpret and use ratios in different contexts (e.g., batting averages, miles RA 

per hour) to show the relative sizes of two quantities, using appropriate 

notations ( a/b, a to b, a:b ). 

1.3 Use proportions to solve problems (e.g., determine the value of N if 4/7 = RNE, SE, RA RN-BT-6 SE-BT-12 

N/ 21 , find the length of a side of a polygon similar to a known polygon). Use RN-BT-2 SE-FS-1 

cross-multiplication as a method for solving such problems, understanding it as RN-EX-6ab SE-BT-20 

the multiplication of both sides of an equation by a multiplicative inverse. RN-BT-5 

RN-BT-6 

RN-EX-18ab 



RN-BT-4 

1.4 Calculate given percentages of quantities and solve problems involving 

discounts at sales, interest earned, and tips. 



2.0 Students calculate and solve problems involving addition, subtraction, RNE, SE, RA RN-BT-7 SE-BT-6 

multiplication, and division: RN-BT-8 SE-BT-7 

RN-BT-9 SE-BT-8 

RN-BT-15 SE-BT-9 

RN-BT-16 SE-BT-12 

RN-BT-17 SE-BT-20 



SE-EX-26 

SE-EX-19ab 

SE-EX-27 

SE-FS-1 

SE-FS-3 



2.1 Solve problems involving addition, subtraction, multiplication, and division RNE, SE, RA RN-EX-6ab SE-EX-19ab 

of positive fractions and explain why a particular operation was used for a given RN-EX-13 SE-EX-26 

situation. RN-BT-15 SE-EX-27 

RN-EX-17 SE-EX-28 

RN-EX-18ab SE-EX-29 

RN-EX-28ab 
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2.2 Explain the meaning of multiplication and division of positive fractions and 
perform the calculations (e.g., 5/8 + 15/16 = 5/8 x 16/15 = 2/3). 


RNE, SE, RA 








2.3 Solve addition, subtraction, multiplication, and division problems, including 
those arising in concrete situations, that use positive and negative integers and 
combinations of these operations. 


RNE, SE, RA 








2.4 Determine the least common multiple and the greatest common divisor of 
whole numbers; use them to solve problems with fractions (e.g., to find a 
common denominator to add two fractions or to find the reduced form for a 
fraction). 




RN-EX-28ab 






Algebra and Functions 










1.0 Students write verbal expressions and sentences as algebraic expressions 
and equations; they evaluate algebraic expressions, solve simple linear 
equations, and graph and interpret their results: 


PA, SE, RA 




SE-WP-1 labc 
SE-WP-12abc 
SE-WP-14 




1.1 Write and solve one-step linear equations in one variable. 


SE, RA 




SE-WP-1 labc, 
SE-WP-12abc 




1.2 Write and evaluate an algebraic expression for a given situation, using up 
to three variables. 


SE, RA 








1.3 Apply algebraic order of operations and the commutative, associative, and 
distributive properties to evaluate expressions; and justify each step in the 
process. 


PA, SE, RA 








1.4 Solve problems manually by using the correct order of operations or by 
using a scientific calculator. 










2.0 Students analyze and use tables, graphs, and rules to solve problems 
involving rates and proportions: 


RA 








2.1 Convert one unit of measurement to another (e.g., from feet to miles, from 
centimeters to inches). 


RA 








2.2 Demonstrate an understanding that rate is a measure of one quantity per 
unit value of another quantity. 


RA 








2.3 Solve problems involving rates, average speed, distance, and time. 


RA 


RN-WP-6, RN-WP-7 






3.0 Students investigate geometric patterns and describe them algebraically: 


RA 








3.1 Use variables in expressions describing geometric quantities (e.g., P = 2w 
+ 21, A = 1/2bh, C = pd - the formulas for the perimeter of a rectangle, the area 
of a triangle, and the circumference of a circle, respectively). 


RA 








3.2 Express in symbolic form simple relationships arising from geometry. 










Measurement and Geometry 










1.0 Students deepen their understanding of the measurement of plane and 
solid shapes and use this understanding to solve problems: 










1.1 Understand the concept of a constant such as p; know the formulas for the 
circumference and area of a circle. 
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1.2 Know common estimates of p (3.14; 22/7) and use these values to estimate 

and calculate the circumference and the area of circles; compare with actual 
measurements. 

1 .3 Know and use the formulas for the volume of triangular prisms and 
cylinders (area of base x height); compare these formulas and explain the 
similarity between them and the formula for the volume of a rectangular solid. 

2.0 Students identify and describe the properties of two-dimensional figures: 

2.1 Identify angles as vertical, adjacent, complementary, or supplementary and 

provide descriptions of these terms. 

2.2 Use the properties of complementary and supplementary angles and the 
sum of the angles of a triangle to solve problems involving an unknown angle. 

2.3 Draw quadrilaterals and triangles from given information about them (e.g., a 
quadrilateral having equal sides but no right angles, a right isosceles triangle). 

Statistics, Data Analysis, and Probability 

1.0 Students compute and analyze statistical measurements for data sets: 

1.1 Compute the range, mean, median, and mode of data sets. 

1.2 Understand how additional data added to data sets may affect these 

computations of measures of central tendency. 

1 .3 Understand how the inclusion or exclusion of outliers affects measures of 

central tendency. 

1.4 Know why a specific measure of central tendency (mean, median) provides 

the most useful information in a given context. 

2.0 Students use data samples of a population and describe the characteristics 

and limitations of the samples: 

2.1 Compare different samples of a population with the data from the entire 
population and identify a situation in which it makes sense to use a sample. 

2.2 Identify different ways of selecting a sample (e.g., convenience sampling, 

responses to a survey, random sampling) and which method makes a sample 
more representative for a population. 

2.3 Analyze data displays and explain why the way in which the question was 
asked might have influenced the results obtained and why the way in which the 
results were displayed might have influenced the conclusions reached. 

2.4 Identify data that represent sampling errors and explain why the sample 

(and the display) might be biased. 

2.5 Identify claims based on statistical data and, in simple cases, evaluate the 
validity of the claims. 



SE, RA 

















6th Grade California Math Standards 



3.0 Students determine theoretical and experimental probabilities and use 

these to make predictions about events: 

3.1 Represent all possible outcomes for compound events in an organized way 

(e.g., tables, grids, tree diagrams) and express the theoretical probability of 
each outcome. 

3.2 Use data to estimate the probability of future events (e.g., batting averages 

or number of accidents per mile driven). 

3.3 Represent probabilities as ratios, proportions, decimals between 0 and 1, 

and percentages between 0 and 100 and verify that the probabilities computed 
are reasonable; know that if P is the probability of an event, 1- P is the 
probability of an event not occurring. 

3.4 Understand that the probability of either of two disjoint events occurring is 
the sum of the two individual probabilities and that the probability of one event 
following another, in independent trials, is the product of the two probabilities. 

3.5 Understand the difference between independent and dependent events. 



Mathematical Reasoning 




1.0 Students make decisions about how to approach problems: 


RNE, PA, SE, 
RA 


1.1 Analyze problems by identifying relationships, distinguishing relevant from 
irrelevant information, identifying missing information, sequencing and 
prioritizing information, and observing patterns. 


RNE, PA, SE, 
RA 


1.2 Formulate and justify mathematical conjectures based on a general 
description of the mathematical question or problem posed. 


RNE, PA, SE, 
RA 


1.3 Determine when and how to break a problem into simpler parts. 


RNE, PA, SE, 
RA 


2.0 Students use strategies, skills, and concepts in finding solutions: 


RNE, PA, SE, 
RA 


2.1 Use estimation to verify the reasonableness of calculated results. 




2.2 Apply strategies and results from simpler problems to more complex 
problems. 


RNE, PA, SE, 
RA 


2.3 Estimate unknown quantities graphically and solve for them by using logical 
reasoning and arithmetic and algebraic techniques. 


SE, RA 



SE-WP-1 labc 
SE-WP-12abc 
SE-FS-1 
SE-FS-3 
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2.4 Use a variety of methods, such as words, numbers, symbols, charts, 
graphs, tables, diagrams, and models, to explain mathematical reasoning. 


RNE, PA, SE, 
RA 


2.5 Express the solution clearly and logically by using the appropriate 
mathematical notation and terms and clear language; support solutions with 
evidence in both verbal and symbolic work. 


RNE, PA, SE, 
RA 


2.6 Indicate the relative advantages of exact and approximate solutions to 
problems and give answers to a specified degree of accuracy. 




2.7 Make precise calculations and check the validity of the results from the 
context of the problem. 


RNE, PA, SE, 
RA 


3.0 Students move beyond a particular problem by generalizing to other 
situations: 


RNE, PA, SE, 
RA 


3.1 Evaluate the reasonableness of the solution in the context of the original 
situation. 


RNE, PA, SE, 
RA 


3.2 Note the method of deriving the solution and demonstrate a conceptual 
understanding of the derivation by solving similar problems. 


RNE, PA, SE, 
RA 


3.3 Develop generalizations of the results obtained and the strategies used and 
apply them in new problem situations. 


RNE, PA, SE, 
RA 



7th Grade Standards 


Domain 


Number Sense 




1.0 Students know the properties of, and compute with, rational numbers 
expressed in a variety of forms: 


RNE, SE, RA 



RN-EX-lab 

RN-EX-6ab 

RN-EX-12 

RN-EX-13 

RN-EX-15 

RN-EX-16ab 

RN-EX-17 

RN-EX-18ab 


SE-EX-19ab 

SE-EX-23ab 

SE-WP-1 labc 

SE-WP-12abc 

SE-FS-1 

SE-FS-3 


RN-EX-16ab 


SE-EX-19ab 

SE-EX-23ab 

SE-EX-26 

SE-EX-28 

SE-WP-1 labc 

SE-WP-12abc 
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1.1 Read, write, and compare rational numbers in scientific notation (positive 
and negative powers of 10) with approximate numbers using scientific notation. 



1.2 Add, subtract, multiply, and divide rational numbers (integers, fractions, and 
terminating decimals) and take positive rational numbers to whole-number 
powers. 



1.3 Convert fractions to decimals and percents and use these representations 
in estimations, computations, and applications. 



1.4 Differentiate between rational and irrational numbers. 



1.5 Know that every rational number is either a terminating or repeating 
decimal and be able to convert terminating decimals into reduced fractions. 



1 .6 Calculate the percentage of increases and decreases of a quantity. 



1.7 Solve problems that involve discounts, markups, commissions, and profit 
and compute simple and compound interest. 



2.0 Students use exponents, powers, and roots and use exponents in working 
with fractions: 



2.1 Understand negative whole-number exponents. Multiply and divide 
expressions involving exponents with a common base. 



2.2 Add and subtract fractions by using factoring to find common 
denominators. 



2.3 Multiply, divide, and simplify rational numbers by using exponent rules. 



2.4 Use the inverse relationship between raising to a power and extracting the 
root of a perfect square integer; for an integer that is not square, determine 
without a calculator the two integers between which its square root lies and 
explain why. 



2.5 Understand the meaning of the absolute value of a number; interpret the 
absolute value as the distance of the number from zero on a number line; and 
determine the absolute value of real numbers. 



Algebra and Functions 



1.0 Students express quantitative relationships by using algebraic terminology, 
expressions, equations, inequalities, and graphs: 



1.1 Use variables and appropriate operations to write an expression, an 
equation, an inequality, or a system of equations or inequalities that represents 
a verbal description (e.g., three less than a number, half as large as area A). 



1.2 Use the correct order of operations to evaluate algebraic expressions such 
as 3(2x + 5)2. 



1.3 Simplify numerical expressions by applying properties of rational numbers 
(e.g., identity, inverse, distributive, associative, commutative) and justify the 
rocess used. 



1.4 Use algebraic terminology (e.g., variable, equation, term, coefficient, 
inequality, expression, constant) correctly. 
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1.5 Represent quantitative relationships graphically and interpret the meaning 
of a specific part of a graph in the situation represented by the graph. 

2.0 Students interpret and evaluate expressions involving integer powers and RNE, PA, RA 

simple roots: 

2.1 Interpret positive whole-number powers as repeated multiplication and RNE, PA, RA 

negative whole-number powers as repeated division or multiplication by the 

multiplicative inverse. Simplify and evaluate expressions that include 

exponents. 

2.2 Multiply and divide monomials; extend the process of taking powers and 
extracting roots to monomials when the latter results in a monomial with an 

integer exponent. 

3.0 Students graph and interpret linear and some nonlinear functions: 

3.1 Graph functions of the form y = nx A 2 and y = nx A 3 and use in solving 

problems. 

3.2 Plot the values from the volumes of three-dimensional shapes for various 
values of the edge lengths (e.g., cubes with varying edge lengths or a triangle 
prism with a fixed height and an equilateral triangle base of varying lengths). 

3.3 Graph linear functions, noting that the vertical change (change in y- value) 
per unit of horizontal change (change in x- value) is always the same and know 
that the ratio ("rise over run") is called the slope of a graph. 

3.4 Plot the values of quantities whose ratios are always the same (e.g., cost to 
the number of an item, feet to inches, circumference to diameter of a circle). Fit 
a line to the plot and understand that the slope of the line equals the quantities. 

4.0 Students solve simple linear equations and inequalities over the rational SE, RA 

numbers: 

4.1 Solve two-step linear equations and inequalities in one variable over the SE, RA 

rational numbers, interpret the solution or solutions in the context from which 

they arose, and verify the reasonableness of the results. 

4.2 Solve multistep problems involving rate, average speed, distance, and time SE, RA 

or a direct variation. 

Measurement and Geometry 

1.0 Students choose appropriate units of measure and use ratios to convert 

within and between measurement systems to solve problems: 

1.1 Compare weights, capacities, geometric measures, times, and 
temperatures within and between measurement systems (e.g., miles per hour 

and feet per second, cubic inches to cubic centimeters). 

1.2 Construct and read drawings and models made to scale. 
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1.3 Use measures expressed as rates (e.g., speed, density) and measures 

expressed as products (e.g., person-days) to solve problems; check the units 

of the solutions; and use dimensional analysis to check the reasonableness of 

the answer. 

2.0 Students compute the perimeter, area, and volume of common geometric RNE, SE, RA 
objects and use the results to find measures of less common objects. They 

know how perimeter, area, and volume are affected by changes of scale: 

2.1 Use formulas routinely for finding the perimeter and area of basic two- SE, RA 

dimensional figures and the surface area and volume of basic three- 

dimensional figures, including rectangles, parallelograms, trapezoids, squares, 

triangles, circles, prisms, and cylinders. 

2.2 Estimate and compute the area of more complex or irregular two-and three- 
dimensional figures by breaking the figures down into more basic geometric 

objects. 

2.3 Compute the length of the perimeter, the surface area of the faces, and the SE, RA 
volume of a three-dimensional object built from rectangular solids. Understand 

that when the lengths of all dimensions are multiplied by a scale factor, the 

surface area is multiplied by the square of the scale factor and the volume is 

multiplied by the cube of the scale factor. 

2.4 Relate the changes in measurement with a change of scale to the units RNE, RA 

used (e.g., square inches, cubic feet) and to conversions between units (1 

square foot = 144 square inches or [1 ft2] = [144 in2], 1 cubic inch is 

approximately 16.38 cubic centimeters or [1 in3j = [16.38 cm3j). 

3.0 Students know the Pythagorean theorem and deepen their understanding 
of plane and solid geometric shapes by constructing figures that meet given 

conditions and by identifying attributes of figures: 

3.1 Identify and construct basic elements of geometric figures (e.g., altitudes, 
mid-points, diagonals, angle bisectors, and perpendicular bisectors; central 
angles, radii, diameters, and chords of circles) by using a compass and 

straightedge. 

3.2 Understand and use coordinate graphs to plot simple figures, determine 
lengths and areas related to them, and determine their image under 

translations and reflections. 

3.3 Know and understand the Pythagorean theorem and its converse and use it 
to find the length of the missing side of a right triangle and the lengths of other 
line segments and, in some situations, empirically verify the Pythagorean 

theorem by direct measurement. 

3.4 Demonstrate an understanding of conditions that indicate two geometrical 
figures are congruent and what congruence means about the relationships 

between the sides and angles of the two figures. 

3.5 Construct two-dimensional patterns for three-dimensional models, such as 

cylinders, prisms, and cones. 
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3.6 Identify elements of three-dimensional geometric objects (e.g., diagonals of 
rectangular solids) and describe how two or more objects are related in space 
(e.g., skew lines, the possible ways three planes might intersect). 



Statistics, Data Analysis, and Probability 




1.0 Students collect, organize, and represent data sets that have one or more 
variables and identify relationships among variables within a data set by hand 
and through the use of an electronic spreadsheet software program: 




1.1 Know various forms of display for data sets, including a stem-and-leaf plot 
or box-and-whisker plot; use the forms to display a single set of data or to 
compare two sets of data. 




1.2 Represent two numerical variables on a scatterplot and informally describe 
how the data points are distributed and any apparent relationship that exists 
between the two variables (e.g., between time spent on homework and grade 
level). 




1.3 Understand the meaning of, and be able to compute, the minimum, the 
lower quartile, the median, the upper quartile, and the maximum of a data set. 




Mathematical Reasoning 




1.0 Students make decisions about how to approach problems: 


RNE, PA, SE, 
RA 


1.1 Analyze problems by identifying relationships, distinguishing relevant from 
irrelevant information, identifying missing information, sequencing and 
prioritizing information, and observing patterns. 


RNE, PA, SE, 
RA 


1.2 Formulate and justify mathematical conjectures based on a general 
description of the mathematical question or problem posed. 


RNE, PA, SE, 
RA 


1.3 Determine when and how to break a problem into simpler parts. 


RNE, PA, SE, 
RA 


2.0 Students use strategies, skills, and concepts in finding solutions: 


RNE, PA, SE, 
RA 


2.1 Use estimation to verify the reasonableness of calculated results. 




2.2 Apply strategies and results from simpler problems to more complex 
problems. 


RNE, PA, SE, 
RA 


2.3 Estimate unknown quantities graphically and solve for them by using logical 
reasoning and arithmetic and algebraic techniques. 




2.4 Make and test conjectures by using both inductive and deductive 
reasoning. 




2.5 Use a variety of methods, such as words, numbers, symbols, charts, 
graphs, tables, diagrams, and models, to explain mathematical reasoning. 


RNE, PA, SE, 
RA 
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2.6 Express the solution clearly and logically by using the appropriate 
mathematical notation and terms and clear language; support solutions with 
evidence in both verbal and symbolic work. 


RNE, PA, SE, 
RA 


2.7 Indicate the relative advantages of exact and approximate solutions to 
problems and give answers to a specified degree of accuracy. 




2.8 Make precise calculations and check the validity of the results from the 
context of the problem. 


RNE, PA, SE, 
RA 


3.0 Students determine a solution is complete and move beyond a particular 
problem by generalizing to other situations: 


RNE, PA, SE, 
RA 


3.1 Evaluate the reasonableness of the solution in the context of the original 
situation. 


RNE, PA, SE, 
RA 


3.2 Note the method of deriving the solution and demonstrate a conceptual 
understanding of the derivation by solving similar problems. 


RNE, PA, SE, 
RA 


3.3 Develop generalizations of the results obtained and the strategies used and 
apply them to new problem situations. 


RNE, PA, SE, 
RA 



8th Grade Standards 


Domain 


Algebra 1 




1 .0 Students identify and use the arithmetic properties of subsets of integers 
and rational, irrational, and real numbers, including closure properties for the 
four basic arithmetic operations where applicable: 




1.1 Students use properties of numbers to demonstrate whether assertions are 
true or false. 


PA, RA 


2.0 Students understand and use such operations as taking the opposite, 
finding the reciprocal, taking a root, and raising to a fractional power. They 
understand and use the rules of exponents. 


RNE, RA 


3.0 Students solve equations and inequalities involving absolute values. 


SE, RA 


4.0 Students simplify expressions before solving linear equations and 
inequalities in one variable, such as 3(2x-5) + 4(x-2) = 12. 


PA, SE, RA 


5.0 Students solve multistep problems, including word problems, involving 
linear equations and linear inequalities in one variable and provide justification 
for each step. 


PA, SE, RA 


6.0 Students graph a linear equation and compute the x- and y- intercepts 
(e.g., graph 2x + 6y = 4). They are also able to sketch the region defined by 
linear inequality (e.g., they sketch the region defined by 2x + 6y < 4). 
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7.0 Students verify that a point lies on a line, given an equation of the line. SE, RA 

Students are able to derive linear equations by using the point-slope formula. 

8.0 Students understand the concepts of parallel lines and perpendicular lines 
and how those slopes are related. Students are able to find the equation of a 
line perpendicular to a given line that passes through a given point. 

9.0 Students solve a system of two linear equations in two variables 
algebraically and are able to interpret the answer graphically. Students are able 
to solve a system of two linear inequalities in two variables and to sketch the 

solution sets. 

10.0 Students add, subtract, multiply, and divide monomials and polynomials. RNE, SE, RA 
Students solve multistep problems, including word problems, by using these 

techniques. 

1 1 .0 Students apply basic factoring techniques to second-and simple third- RNE, RA 

degree polynomials. These techniques include finding a common factor for all 

terms in a polynomial, recognizing the difference of two squares, and 

recognizing perfect squares of binomials. 

12.0 Students simplify fractions with polynomials in the numerator and RNE, RA 

denominator by factoring both and reducing them to the lowest terms. 

13.0 Students add, subtract, multiply, and divide rational expressions and RNE, SE, RA 

functions. Students solve both computationally and conceptually challenging 

problems by using these techniques. 

14.0 Students solve a quadratic equation by factoring or completing the square. 

15.0 Students apply algebraic techniques to solve rate problems, work SE, RA 

problems, and percent mixture problems. 

16.0 Students understand the concepts of a relation and a function, determine 
whether a given relation defines a function, and give pertinent information 

about given relations and functions. 

17.0 Students determine the domain of independent variables and the range of 
dependent variables defined by a graph, a set of ordered pairs, or a symbolic 

expression. 

18.0 Students determine whether a relation defined by a graph, a set of 
ordered pairs, or a symbolic expression is a function and justify the conclusion. 

19.0 Students know the quadratic formula and are familiar with its proof by 

completing the square. 

20.0 Students use the quadratic formula to find the roots of a second-degree 

polynomial and to solve quadratic equations. 

21 .0 Students graph quadratic functions and know that their roots are the x- 
intercepts. 
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22.0 Students use the quadratic formula or factoring techniques or both to 
determine whether the graph of a quadratic function will intersect the x-axis in 

zero, one, or two points. 

23.0 Students apply quadratic equations to physical problems, such as the 

motion of an object under the force of gravity. 

24.0 Students use and know simple aspects of a logical argument: 

24.1 Students explain the difference between inductive and deductive 

reasoning and identify and provide examples of each. 

24.2 Students identify the hypothesis and conclusion in logical deduction. 24.3 
Students use counterexamples to show that an assertion is false and recognize 

that a single counterexample is sufficient to refute an assertion. 

25.0 Students use properties of the number system to judge the validity of PA, RA 

results, to justify each step of a procedure, and to prove or disprove 

statements: 

25.1 Students use properties of numbers to construct simple, valid arguments PA, RA 
(direct and indirect) for, or formulate counterexamples to, claimed assertions. 

25.2 Students judge the validity of an argument according to whether the PA, RA 

properties of the real number system and the order of operations have been 

applied correctly at each step. 

25.3 Given a specific algebraic statement involving linear, quadratic, or 
absolute value expressions or equations or inequalities, students determine 

whether the statement is true sometimes, always, or never. 
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Appendix G: 
Teacher Surveys 




Teacher Background Survey Questions 



Teacher ID: 

1. What is your age? (in years) 

2. Are you female or male? (Circle one) 

Female Male 

3. Please select the one title that best describes your current position. 

□ Classroom teacher (1-2 years math teaching experience) 

□ Classroom teacher (3 or more years math teaching experience) 

□ Out-of-classroom position such as math coach or district math personnel 

□ Other, please specify 

4. As of the end of the 2006-2007 school year, how many years will you have been 
teaching? 

Total number of years teaching (any subject) 

Total number of years teaching math 

Total number of years teaching middle school 

math 

5. Do you have National Board Certification (please circle one) 

Yes No 

If yes, in what area? 

6. Have you completed a Bachelor's degree? (circle one) 

Yes No 

If yes, in what fields(s). Please specify major(s) and minor(s) 



7. Have you completed a Master's degree? (circle one) 

Yes No 

If yes, in what fields(s). If yes, in what field(s)? Please specify specialization(s). For 
example, a Master's degree in education might be in curriculum/ instruction or 
administration. 
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8. What type of teaching credentials(s) do you hold? (Check all that apply) 

□ Clear Credential 

□ Preliminary Credential 

□ Emergency Credential 

□ Multiple Subjects 

□ Single Subject (math) 

□ Single subject (other than math) 

□ Currently in a program to obtain my teaching credential 

□ None 

□ Other, please specify 

9. How many undergraduate or graduate courses have you taken at a college or 
university in the following areas? 

Mathematics courses 

Methods of teaching mathematics courses 

Education courses 

10. Considering your training and experience in both mathematics content and 
instruction, how prepared do you feel to teach these topics? 

(Please check one box for each topic) 





1 

Not 

Prepared 


2 

Prepared 


3 

Very 

Prepared 


Additive identity 








Distributive property 








Equivalent fractions 








Multiplicative identity 








Proportions 








Rational numbers 








Simplifying 








Solving equations (one variable) 








Solving equations (two variables) 









11. Compared to the average math teacher at the grade level you teach/ taught, how 
would you rate your level of knowledge of the following math content areas? (Please 
check one box for each content area.) 





1 

Very 

Low 


2 

Low 


3 

Average 


4 

High 


5 

Very 

High 


Equivalence of rational 
numbers 












Distributive property 












Principles for solving 
equations 
















12. Please rate your level of expertise in each of the items below. For each item check ONE 
(none, novice, adequate, good, expert) 





1 

none 


2 

novice 


3 

adequate 


4 

good 


5 

expert 


Justify the distributive 
property for natural 
numbers by using 
multiplication as repeated 
addition 












Explain why when adding 
fractions with common 
denominators, the 
numerators are added and 
the denominators stay the 
same. 












Explain why any non-zero 
number divided by it self is 
1, and how this can be used 
with the multiplicative 
identity to scale ratios. 












Explain how the distributive 
law is used to simplify the 
sum of many linear, single 
variable equations. 












Explain how to solve a 
simplified, linear, single 
variable equation by using 
inverses of arithmetic 
operations and equivalence 
operations on both sides. 












Solving an equation means 
finding all possible values of 
given variables that make 
the equation true. 












Multiplying any number by 
1, the multiplicative identity, 
results in a product that is 
the original number. 












Equivalence is a 
fundamental property of 
rational numbers: equivalent 
fractions, percents, and 
decimals all name the same 
relationship between two 
values. 
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13. How often do you have the following types of interactions with other teachers? (Fill in 
check on box for each topic.) 





1 

Never or 
almost 
never 


2 

A few 
times per 
month 


3 

A few 
times per 
week 


4 

Daily or 
almost 
daily 


Discussion about how to 
teach a particular 
concept 










Working on preparing 
instructional materials 










Visits to another 
teacher's classroom to 
observe his/her teaching 










Informal observations of 
my classroom by 
another teacher 











14. In the past three years, have you participated in ay of the following general 
professional development trainings? (Please check all that apply) 

□ Mathematics content 

□ Mathematics pedagogy/ instruction 

□ Mathematics curriculum 

□ Integrating information technology into mathematics 

□ Improving students' critical thinking or problem solving skills 

□ None 

□ Other, please specify 



15. In the past three years, have you participated in any of the following professional 
development trainings about assessment? (Please check all that apply) 

□ Using student assessment to evaluate instructional effectiveness 

□ Using student assessment to evaluate student learning 

□ Using student assessment to tailor instruction to students' skill level 

□ Learning about standards-based assessment 

□ None 

□ Other, please specify 

16 . How many total hours have you spent on in-service/ professional development 
education in the 2006-2007 school years? Include attendance at such things as 
professional meetings, workshops, and conferences as well as any formal courses for 
which you have received college credit. 
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17. Please use 8 hours as the standard for a full day in-service and 40 hours as the standard for a 
week-long all day in-service or training. 

Math content in-service hours 

Methods for teaching math in- service hours 

Other math topics in-service 



18. To what extent do you agree or disagree with each of the following statements? 
(Please check one box for each statement) 





1 

Disagree 
a lot 


2 

Disagree 


3 

Agree 


4 

Agree a 
lot 


More than one representation 
(picture, concrete material, 
symbols, etc.) must be used in 
teaching a mathematics topic 










Mathematics should be learned 
as sets of algorithms or rules 
that cover all possibilities 










Solving mathematics problems 
often involves hypothesizing, 
estimating, testing, and 
modifying findings 










Learning mathematics mainly 
involves memorizing 










There are different ways to 
solve most mathematical 
problems 










Few new discoveries in 
mathematics are being made 










Modeling real-world problems 
is essential to teaching 
mathematics 
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19. Do you use a textbook(s) with your math class? (Circle one) 

Yes No 

If yes, please write 1) publisher and 2) the hie of each book you use. If you use more 
than one textbook, please write the primary textbook first. 



20. How do you use a textbook (s) in teaching mathematics to you math class (Please 
check one) 

□ As the primary basis for my lessons 

□ As supplementary resources 

□ Other, please specify, 



21. How often do your students do the following activities? 
(Please check one box for each statement) 





1 

Never 


2 

Rarely 


3 

Weekly 


4 

Daily 


Reviewing/ grading 
homework 










Listening to lecture-style 
presentations 










Working problems with 
your guidance 










Working problems on their 
own without your guidance 










Listening to you re-teach 
and clarify 
content/ procedures 










Taking tests or quizzes 










Working in groups to solve 
problems 
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22. How often do you include the following types of questions in your mathematics tests 
of examinations? (Please check one box for each statement) 





1 

Always/ 

almost 

always 


2 

Sometimes 


3 

Never or 
almost 
never 


Questions involving 
application of mathematical 
procedures 








Questions involving 
searching for patterns and 
relationships 








Questions requiring written 
explanation or justifications 









23. Please indicate your opinion about each of the statements below relating to the kinds 
of things that may create difficulties for teachers in their school activities. (Please check 
one box for each statement) 





1 


2 


3 


4 


5 


6 


7 


8 


9 




Not at all 




Very 

Little 




Some 

Influence 




Quite a 
Bit 




A Great 
Deal 


1. To what extent can you 
use a variety of assessment 
strategies? 




















2. To what extent can you 
provide an alternative 
explanation or example 
when students are confused? 




















3. To what extent can you 
craft good questions for 
your students? 




















4. How well can you 
implement alternative 
strategies in your 
classroom? 




















5. How well can you 
respond to difficult 
questions from your 
students? 





















7 






6. How much can you do to 
adjust your lessons to the 
proper level for individual 
students? 




















7. To what extent can you 
gauge student 

comprehension of what you 
have taught? 




















8. How well can you 
provide appropriate 
challenges for very capable 
students? 




















17. How much can you do 
to get students to believe 
they can do well in 
schoolwork? 




















18. How much can you do 
to help your students’ value 
learning? 




















19. How much can you do 
to motivate students who 
show low interest in 
schoolwork? 




















20. How much can you 
assist families in helping 
their children do well in 
school? 




















2 1 . How much can you do 
to improve the 
understanding of a student 
who is failing? 




















22. How much can you do 
to help your students think 
critically? 




















23. How much can you do 
to foster student creativity? 




















24. How much can you do 
to get through to the most 
difficult students? 























24. Other comments/ questions 



Thank you for completing the survey! 
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Teacher Background Survey Questions 



Teacher ID: 

1. Considering your training and experience in both mathematics content and 
instruction, how prepared do you feel to teach these topics? 

(Please check one box for each topic) 





1 

Not 

Prepared 


2 

Prepared 


3 

Very 

Prepared 


Additive identity 








Distributive property 








Equivalent fractions 








Multiplicative identity 








Proportions 








Rational numbers 








Simplifying 








Solving equations (one variable) 








Solving equations (two variables) 









2. Compared to the average math teacher at the grade level you teach/ taught, how 
would you rate your level of knowledge of the following math content areas? (Please 
check one box for each content area.) 





1 

Very 

Low 


2 

Low 


3 

Average 


4 

High 


5 

Very 

High 


Equivalence of rational 
numbers 












Distributive property 












Principles for solving 
equations 
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3. Please rate your level of expertise in each of the items below. For each item check ONE 
(none, novice, adequate, good, expert) 





1 

none 


2 

novice 


3 

adequate 


4 

good 


5 

expert 


Justify the distributive 
property for natural 
numbers by using 
multiplication as repeated 
addition 












Explain why when adding 
fractions with common 
denominators, the 
numerators are added and 
the denominators stay the 
same. 












Explain why any non-zero 
number divided by it self is 
1, and how this can be used 
with the multiplicative 
identity to scale ratios. 












Explain how the distributive 
law is used to simplify the 
sum of many linear, single 
variable equations. 












Explain how to solve a 
simplified, linear, single 
variable equation by using 
inverses of arithmetic 
operations and equivalence 
operations on both sides. 












Solving an equation means 
finding all possible values of 
given variables that make 
the equation true. 












Multiplying any number by 
1, the multiplicative identity, 
results in a product that is 
the original number. 












Equivalence is a 
fundamental property of 
rational numbers: equivalent 
fractions, percents, and 
decimals all name the same 
relationship between two 
values. 
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4. How often do you have the following types of interactions with other teachers? (Fill 
check on box for each topic.) 





1 

Never or 
almost 
never 


2 

A few 
times per 
month 


3 

A few 
times per 
week 


4 

Daily or 
almost 
daily 


Discussion about how to 
teach a particular 
concept 










Working on preparing 
instructional materials 










Visits to another 
teacher's classroom to 
observe his/her teaching 










Informal observations of 
my classroom by 
another teacher 











5. To what extent do you agree or disagree with each of the following statements? 
(Please check one box for each statement) 





1 

Disagree 
a lot 


2 

Disagree 


3 

Agree 


4 

Agree a 
lot 


More than one representation 
(picture, concrete material, 
symbols, etc.) must be used in 
teaching a mathematics topic 










Mathematics should be learned 
as sets of algorithms or rules 
that cover all possibilities 










Solving mathematics problems 
often involves hypothesizing, 
estimating, testing, and 
modifying findings 










Learning mathematics mainly 
involves memorizing 










There are different ways to 
solve most mathematical 
problems 














Few new discoveries in 
mathematics are being made 










Modeling real-world problems 
is essential to teaching 
mathematics 











6. How often do your students do the following activities? 
(Please check one box for each statement) 





1 

Never 


2 

Rarely 


3 

Weekly 


4 

Daily 


Reviewing/ grading 
homework 










Listening to lecture-style 
presentations 










Working problems with 
your guidance 










Working problems on their 
own without your guidance 










Listening to you re-teach 
and clarify 
content/ procedures 










Taking tests or quizzes 










Working in groups to solve 
problems 











7. How often do you include the following types of questions in your mathematics tests 
of examinations? (Please check one box for each statement) 





1 


2 


3 




Always/ 

almost 


Sometimes 


Never or 
almost 




always 




never 


Questions involving 
application of mathematical 
procedures 








Questions involving 
searching for patterns and 
relationships 
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Questions requiring written 
explanation or justifications 









8. Please indicate your opinion about each of the statements below relating to the kinds 
of things that may create difficulties for teachers in their school activities. (Please check 
one box for each statement) 





1 


2 


3 


4 


5 


6 


7 


8 


9 




Not at all 




Very 

Little 




Some 

Influence 




Quite a 
Bit 




A Great 
Deal 


1. To what extent can you 
use a variety of assessment 
strategies? 




















2. To what extent can you 
provide an alternative 
explanation or example 
when students are confused? 




















3. To what extent can you 
craft good questions for 
your students? 




















4. How well can you 
implement alternative 
strategies in your 
classroom? 




















5. How well can you 
respond to difficult 
questions from your 
students? 




















6. How much can you do to 
adjust your lessons to the 
proper level for individual 
students? 




















7. To what extent can you 
gauge student 

comprehension of what you 
have taught? 




















8. How well can you 
provide appropriate 
challenges for very capable 
students? 
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17. How much can you do 
to get students to believe 
they can do well in 
schoolwork? 




















18. How much can you do 
to help your students’ value 
learning? 




















19. How much can you do 
to motivate students who 
show low interest in 
schoolwork? 




















20. How much can you 
assist families in helping 
their children do well in 
school? 




















2 1 . How much can you do 
to improve the 
understanding of a student 
who is failing? 




















22. How much can you do 
to help your students think 
critically? 




















23. How much can you do 
to foster student creativity? 




















24. How much can you do 
to get through to the most 
difficult students? 





















9 . Other comments / questions 



Thank you for completing the survey! 
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Teacher Survey- 1 7/3/2008 

Teacher Survey-Rational Number Equivalence 
Group 1 



1. Teacher I D: 



2. When you gave the first Checks for Understanding to your students, had 
you already covered any of the concepts addressed within (in your regular 
school curriculum)? 

Yes No 



3. Briefly explain (if applicable) how you changed your instruction on rational 
number equivalence after administering, and reviewing the Checks for 
Understanding : 



4. Did you feel that you had enough time to complete all that was asked of 
you? 

Yes No 



5. Did you use lesson 1 in the Teacher Handbook! Yes No 



6. If yes, please rate how closely you adhered to the sample lesson presented 
in the Teacher Handbook: 

Not at all Somewhat Very closely 

1 2 3 4 5 
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Teacher Survey- 1 



7/3/2008 



7. Did you use lesson 2 in the Teacher Handbook! Yes No 



8. If you used the lessons, approximately how long did each one of them 
take? (in minutes) 

Lesson 1: 

Lesson 2: 

9. If applicable, please, briefly, describe how you used Lesson 2 in the 
Teacher Handbook. 



10. What, if, any difficulties did you encounter in using either the Checks for 
Understanding or the lessons in the Teacher Handbook! (please describe) 



11. If you used them, did you feel as if the lessons in the Teacher Handbook 
increased your students' understanding of the topic? 



12. Do you have any suggestions about how any of the materials (assessments, 
Lessons, teacher instructions) could be improved? 
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Teacher Survey- 1 



7/3/2008 



13. Having taught rational number equivalence, how confident are you that 
your students understand the concepts? 

Not at all Somewhat Very 

1 2 3 4 5 



THANK YOU FOR COMPLETI NG THI S SURVEY! 

If you have any comments about the questions, please write them in the space 

below. 
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Teacher Survey- 1 7/3/2008 

Teacher Survey-Properties of Arithmetic 
Group 1 



1. Teacher I D: 



2. When you gave the first Checks for Understanding to your students, had 
you already covered any of the concepts addressed within (in your regular 
school curriculum)? 

Yes No 



3. Briefly explain (if applicable) how you changed your instruction on the 
properties of arithmetic— focus on the distributive property— after 
administering, and reviewing the Checks for Understanding: 



4. Did you feel that you had enough time to complete all that was asked of 
you? 

Yes No 



5. Did you use lesson 1 in the Teacher Handbook! Yes No 



6. If yes, please rate how closely you adhered to the sample lesson presented 
in the Teacher Handbook: 

Not at all Somewhat Very closely 

1 2 3 4 5 
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Teacher Survey- 1 



7/3/2008 



7. Did you use lesson 2 in the Teacher Handbook! Yes No 



8. If you used the lessons, approximately how long did each one of them 
take? (in minutes) 

Lesson 1: 

Lesson 2: 

9. If applicable, please, briefly, describe how you used Lesson 2 in the 
Teacher Handbook. 



10. What, if, any difficulties did you encounter in using either the Checks for 
Understanding or the lessons in the Teacher Handbook! (please describe) 



11. If you used them, did you feel as if the lessons in the Teacher Handbook 
increased your students' understanding of the topic? 



12. Do you have any suggestions about how any of the materials (assessments, 
Lessons, teacher instructions) could be improved? 
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Teacher Survey- 1 



7/3/2008 



13. Having taught the distributive property, how confident are you that your 
students understand the concepts? 

Not at all Somewhat Very 

1 2 3 4 5 



THANK YOU FOR COMPLETI NG THI S SURVEY! 

If you have any comments about the questions, please write them in the space 

below. 
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Teacher Survey- 1 7/3/2008 

Teacher Survey-Solving Equations 
Group 1 



1. Teacher I D: 



2. When you gave the first Checks for Understanding to your students, had 
you already covered any of the concepts addressed within (in your regular 
school curriculum)? 

Yes No 



3. Briefly explain (if applicable) how you changed your instruction on the 
properties of arithmetic— focus on the distributive property— after 
administering, and reviewing the Checks for Understanding: 



4. Did you feel that you had enough time to complete all that was asked of 
you? 

Yes No 



5. Did you use lesson 1 in the Teacher Handbook! Yes No 



6. If yes, please rate how closely you adhered to the sample lesson presented 
in the Teacher Handbook: 

Not at all Somewhat Very closely 

1 2 3 4 5 
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Teacher Survey- 1 



7/3/2008 



7. Did you use lesson 2 in the Teacher Handbook! Yes No 



8. If you used the lessons, approximately how long did each one of them 
take? (in minutes) 

Lesson 1: 

Lesson 2: 

9. If applicable, please, briefly, describe how you used Lesson 2 in the 
Teacher Handbook. 



10. What, if, any difficulties did you encounter in using either the Checks for 
Understanding or the lessons in the Teacher Handbook! (please describe) 



11. If you used them, did you feel as if the lessons in the Teacher Handbook 
increased your students' understanding of the topic? 



12. Do you have any suggestions about how any of the materials (assessments, 
Lessons, teacher instructions) could be improved? 
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Teacher Survey- 1 



7/3/2008 



13. Having taught the distributive property, how confident are you that your 
students understand the concepts? 

Not at all Somewhat Very 

1 2 3 4 5 



THANK YOU FOR COMPLETI NG THI S SURVEY! 

If you have any comments about the questions, please write them in the space 

below. 
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Teacher Survey- 1 



Teacher Survey - Review & Applications 
Group 1 



7/3/2008 



1. Teacher I D: 



2. When you gave the first Checks for Understanding to your students, had 
you already covered any of the concepts addressed within (in your regular 
school curriculum)? 

Yes No 



3. Briefly explain (if applicable) how you changed your instruction on the 
concepts found within the review and applications handbook after 
administering, and reviewing the Checks for Understanding: 



4. Did you feel that you had enough time to complete all that was asked of 
you? 

Yes No 



5. Did you use lesson 1 in the Teacher Handbook! Yes No 



6. If yes, please rate how closely you adhered to the sample lesson presented 
in the Teacher Handbook: 

Not at all Somewhat Very closely 

1 2 3 4 5 
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Teacher Survey- 1 



7/3/2008 



7. Did you use lesson 2 in the Teacher Handbook! Yes No 



8. If you used the lessons, approximately how long did each one of them 
take? (in minutes) 

Lesson 1: 

Lesson 2: 

9. If applicable, please, briefly, describe how you used Lesson 2 in the 
Teacher Handbook. 



10. What, if, any difficulties did you encounter in using either the Checks for 
Understanding or the lessons in the Teacher Handbook! (please describe) 



11. If you used them, did you feel as if the lessons in the Teacher Handbook 
increased your students' understanding of the topic? 



12. Do you have any suggestions about how any of the materials (assessments, 
Lessons, teacher instructions) could be improved? 
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Teacher Survey- 1 



7/3/2008 



13. Having taught the material in the review and applications handbook, how 
confident are you that your students understand the concepts? 

Not at all Somewhat Very 

1 2 3 4 5 



THANK YOU FOR COMPLETI NG THI S SURVEY! 

If you have any comments about the questions, please write them in the space 

below. 
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Appendix H: 

Interview and Observation Measures 




Teacher Interview Protocol 
2007/2008 




Introductory Remarks: 

Thank you for taking the time to participate in the classroom observation and teacher interview study. This 
interview will take approximately 20-30 minutes to complete. I just want to remind you again that this interview 
is being recorded and I want to make sure that this is still okay with you. 

As you may have already heard, we are conducting these interviews with teachers who participated in the 
UCLA PowerSource/CRESST Study. The information we gather will help members of the research team 
improve professional development trainings and materials for the 2008-2009 school year. Your responses will 
be kept confidential and will be used for this purpose. Do you have any questions before we begin? 



Teacher Interview Measure -V2. 1 Revised 05. 19.2008 





Teacher 



I. Implmentation of Big Ideas/Curricullum 



Content Covered Check for each time period 
content area is addressed at 5 minute intervals 


Example 


Domain 


Frequency (in m 


5 


15 


25 


35 


"It (a number or variable) goes away" 




SE 


□ 


□ 


□ 


□ 


"Whatever you do to one side..." 




SE 


□ 


□ 


□ 


□ 


Additive Identity 


a + 0 = a 


SE 


□ 


□ 


□ 


□ 


Additive Inverse 


a + -a = 0 


SE 


□ 


□ 


□ 


□ 


Commutative Properties 


a + b = b + a 


SE 


□ 


□ 


□ 


□ 


Inverse Operations 


e.g. addition is 
inverse of 
subtraction 


SE 


□ 


□ 


□ 


□ 


Properties of Equality (addition, 
multiplication) 




SE 


□ 


□ 


□ 


□ 


Any non-zero number divided by itself is 1 


a/a = 1 (a ? 0) 


RNE 


□ 


□ 


□ 


□ 


Cross Multiply 




RNE 


□ 


□ 


□ 


□ 


Multiply the "top & bottom" by the same 
number 


'Ax 2 = 2/4 


RNE 


□ 


□ 


□ 


□ 


Area/Array model 


*** 

*** 


PA 


□ 


□ 


□ 


□ 


Distribute multiplier to each addend 




PA 


□ 


□ 


□ 


□ 


Repeated Addition model 




PA 


□ 


□ 


□ 


□ 


"Cancel" 




RNE/SE 


□ 


□ 


□ 


□ 


Multiplicative Identity 


x • 1 = x, x * 0 


RNE/SE 


□ 


□ 


□ 


□ 


Multiplicative Inverse 




RNE/SE 


□ 


□ 


□ 


□ 


Multiply by a number 




RNE/SE 


□ 


□ 


□ 


□ 


Division is the same as multiplication by the 
reciprocal (invert & multiply) 






□ 


□ 


□ 


□ 



Other 



□ □ □ □ 



TOTAL 



Observation 

















































































































































































II. Teacher Behaviors 



Please indicate if activity is occuring (at 
5 minute intervals) 


Frequency (in minutes) 




5 


15 


25 


35 


45 


55 


Teacher discusses misconceptions based on 
PowerSource results 


□ 


□ 


□ 


□ 


□ 


□ 




Teacher demonstrates correct solution 
methods for a problem 


□ 


□ 


□ 


□ 


□ 


□ 




Teacher demonstrates incorrect solution 
methods for a problem 


□ 


□ 


□ 


□ 


□ 


□ 




Teacher provides procedural feedback (i.e., 
answer is right or wrong) 


□ 


□ 


□ 


□ 


□ 


□ 




Teacher provides procedural feedback plus 
correction (i.e., answer is right or wrong and 
correct answer) 


□ 


□ 


□ 


□ 


□ 


□ 




Teacher questions for understanding (i.e., 
teacher uses questioning to help better 
understand reasons for a student's answer) 


□ 


□ 


□ 


□ 


□ 


□ 




Teacher uses guided problem solving (i.e., 
uses questions to help students find the 


□ 


□ 


□ 


□ 


□ 


□ 





correct responses themselves) 














Teacher specifically references Checks for 
Understanding results 


□ 


□ 


□ 


□ 


□ 


□ 


Teacher specifically references other 
test/quiz results 


□ 


□ 


□ 


□ 


□ 


□ 


Teacher specifically references homework 
results 


□ 


□ 


□ 


□ 


□ 


□ 

















TOTAL 



Observation 



Comments 



















































































Teacher 



III. Lesson Structure 



Please estimate percentage of time spent on the 
following activities 


Percentage (%) 


Individual work (students working independently) 


% 


Small groups (3+ students) 


% 


Pairs 


% 


Whole group/teacher leactures to class 


% 


Whole group/teacher leads class in discussion 


% 


Whole group/teacher questions students 


% 


Whole group/teacher provides feedback to students 


% 


Other (list) 


% 



Observation 
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Comments 

























Teacher 



|lV. Use of PowerSource and other materials 


Please indicate if PowerSource of other materials were utilized 


Yes (check) 


Teacher used lesson from handbook directly as written 


□ 


Teacher integrated handbook lesson with other materials (e.g., self- 
designed, other curriculum) 


□ 


Teacher did not use lesson from handbook 


□ 


Did the teacher cover all concepts in the handbook? 


□ 


Did the teacher cover the concepts in the same order as presented in the 
handbook? 


□ 


Which of the following materials were used? 




- Lesson 2 Worksheet 


□ 


- Tally sheets 


□ 


- Powerpoint slides (PowerSource)/overheads 


□ 


- Powerpoint slides (non-PowerSource)/overheads 


□ 


- Manipulatives 


□ 


- Computers/handhelds 


□ 


- Other (please describe) 


□ 



Comments 



































Teacher Classroom Observation 
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Formative Assessments 


1(a). How did you use the results from the 1 st Checks for Understanding to plan your instruction for Lesson 1? 




1(b). How did you use the results of the 2" a Checks for Understanding to plan Lesson 2? 




2(a). What student strenaths emeraed from (understand™ skills and knowledael the 1 SI Check for Understand™? 




2(b). Are there other sources of data (such as other quizzes tests other formative assessments homework or reports from tutors) you use to 
determine where vour students' strenaths lie? 




3(a). What learnina challenaes emerged from the results of the 2"“ Check for Understanding? 




3(b). Are there other sources of data (such as other quizzes tests other formative assessments homework or reports from tutors) you use to 
determine where vour students' areas of difficultv lie? 




4. What strategies do you think helps student’s grasp Big ideas? (instruction of area/array, repeated addition model, other procedural or 
conceptual knowledge)? 
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Instructional Plan Development 


(5). What specifically do you use to develop your instruction plan? 




(6). Do vou find the PowerSource materials easv/hard to use? (Handbooks assessments overheads)? (interviewer include oromots to helo 
teacher expand and understand nature of the questions) 




7. How much time did you use preparing for this PowerSource lesson? 




8. How much time do you use to develop an average (non-PowerSource) lesson? 




Professional Development 


9. What makes PowerSource materials used in the professional development trainings easy (hard) to use? (PD content, format, lesson 
organization, handbook examples, overheads, forms)? 




10. Do you have any specific recommendations for improving/modifying the PowerSource materials? 




1 1 . How well do you feel PowerSource materials align with existing district curricula? 







12. What knowledge from Professional Development trainings helped you implement Big Ideas in the classroom? (research, instructional 
methods for Big Ideas, scoring rubrics, common student errors, comparable items on Checks for Understanding). 



13. What information from the Professional Development trainings helped you use assessment information in a formative way? (discussion of 
student responses, frequencies, percents, common student errors, scoring rubrics, instructional methods for Big Ideas, research behind Big 
Ideas, comparable items on Checks for Understanding). 



14. What do you feel would improve the Professional Development T rainings? 



15. What or how well does Professional Development content align(s) with existing district curricula? 



16. How well prepared do you feel implementing PowerSource and why? What contributes to your preparedness/lack of preparedness? 



Additional Comments 

17. Is there anything else that you would like to add or mention that you did not get a chance to say? 
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